QR CODE LOCALIZATION USING DEEP NEURAL NETWORKS

Size: px
Start display at page:

Download "QR CODE LOCALIZATION USING DEEP NEURAL NETWORKS"

Transcription

1 2014 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT , 2014, REIMS, FRANCE QR CODE LOCALIZATION USING DEEP NEURAL NETWORKS Tamás Grósz Péter Bodnár László Tóth László G. Nyúl MTA-SZTE Research Group on Artificial Intelligence Hungarian Academy of Sciences and University of Szeged Department of Image Processing and Computer Graphics, University of Szeged {groszt, bodnaar, tothl, ABSTRACT Usage of computer-readable visual codes became common in our everyday life at industrial environments and private use. The reading process of visual codes consists of two steps, localization and data decoding. This paper introduces a new method for QR code localization using conventional and deep rectifier neural networks. The structure of the neural networks, regularization, and training parameters, like input vector properties, amount of overlapping at samples, and effect of different block sizes are evaluated and discussed. Results are compared to localization algorithms of the literature. Index Terms QR code, Object detection, Pattern recognition, Neural networks, Machine learning 1. INTRODUCTION QR code is a common type of visual code format that is used at various industrial setups and private projects as well. Its structure is well-defined and makes automatic reading available by computers and embedded systems. QR codes had a decent increase of usage in the last couple of years, more than other patented code types, like Aztec codes or Maxicodes. This is due to its well-constructed error correction scheme that allows recovery of damaged codes up to cca. % of damage. Image acquisition techniques and computer hardware have also improved significantly, that made automatic reading of QR codes available. State of the art algorithms do not require human presence and assumptions on code orientation, position and coverage rate in the image any longer. However, image quality and acquisition techniques vary considerably and each application has its own requirements for detection speed and accuracy, making the task more complex. The recognition process consists of two steps, localization and decoding. The literature already has a wide selection of papers proposing algorithms for efficient QR code localization [1 5], however, each has its own strengths and weaknesses. For example, while these methods are proven to be accurate, morphological operations, convolutions, corner detection and calculation of convexity defects can be a bottleneck for processing performance. Applications using various machine learning techniques can overcome this issue and produce efficient solutions respect to speed and accuracy. In the last few years, there has been a renewed interest in applying neural networks especially deep neural networks to various tasks. As the name suggests, deep neural networks (DNN) differ from conventional ones in that they consist of several hidden layers. However, to properly train these deep networks, the training method requires modifications as the conventional backpropagation algorithm encounters difficulties ( vanishing gradient and explaining away effects). In this case the vanishing gradient effect means that the error might vanish as it gets propagated back through the hidden layers [6]. In this way some hidden layers, in particular those that are close to the input layer, may fail to learn during training. At the same time, in fully connected deep networks, the explaining away effects make inference extremely difficult in practice [7]. Several solutions have been proposed to counter these problems. These solutions modify either the training algorithm by extending it with a pretraining phase [7, 8], or the architecture of the neural networks [9]. 2. THE PROPOSED METHOD The first step of the localization process is uniform partitioning of the image into square blocks. Each block is processed by the neural network individually and a measure is assigned which reflects the probability of presence of a QR code part in that block. After the evaluation of all image blocks, a matrix is formed with the probability values (Fig. 1). The next step of the process is to find clusters in this matrix that has sufficient size, compactness and high values of probability to form a QR code. The final step is to return the cluster centers that satisfy these conditions, and give bounding boxes for the probable QR code areas. For each block the image is divided into, a one-dimensional vector is formed by reading pixels in a circular pattern as sug /14/$31.00 c 2014 IEEE

2 (a) example real-life image (b) feature image Fig. 1. Image captured by a phone camera and visualized feature image according to the output of the neural network. gested in [10]. This pattern provides prominent features while keeping the number of necessary pixels low, typically on 6-10 % of the image pixels. Furthermore, it can indicate presence of a QR code in any orientation The Neural Network Vectors are passed to the core of this approach, which is the neural network. In this paper, both conventional and Deep Rectifier Neural Networks (DRN) have been evaluated. DRNs alter the hidden neurons in the network and not the training algorithm by using rectified linear units. These rectified units differ from standard neurons only in their activation function, as they apply the rectifier function (max(0, x)) instead of the sigmoid or hyperbolic tangent activation. Owing to their properties, the DRN does not require any pre-training to achieve good results [9]. The rectifier function has two important properties, namely its hard saturation at 0 and its linear behaviour for positive input. The first property means that only a subset of neurons will be active in each hidden layer. For example, when we initialize the weights uniformly, around half of the hidden units output are zeros. In theory, this hard saturation at 0 could harm optimization by blocking gradient backpropagation. Fortunately, experimental results do not support this, showing that the hard non-linearities do no harm as long as the gradient can propagate along some path [9]. Owing to the other property of the rectified units, namely the linear behaviour of the active units, there is no vanishing gradient effect [9]. This linear behaviour also means that the computational cost will be smaller, as there is no need to compute the exponential function during the activation calculation, and the sparsity can also be exploited. Unfortunately, there is a disadvantage because of this linearity, the exploding gradient effect, which means that the gradients can grow without limit. To prevent this, we applied L1 normalization by scaling the weights such that the L1 norm of each layer s weights remained the same as it was after initialization. What makes this possible is that for a given input the subset of active neurons behaves linearly, so a scaling of the weights is equivalent to a scaling of the activations. Our deep networks consisted of three hidden layers and each hidden layer had 1000 rectified neurons, as DRN with this structure yielded the best results on the development sets. The shallow neural net was a sigmoid net with only one hidden layer, with the same number of hidden neurons (00) as that for the deep one. The output layer of the neural networks consisted of two softmax neurons, one for the positive and one for the negative label, allowing the networks to output not only classification decisions but also posterior probabilities. As error function we applied the cross entropy function. In our study were utilized two regularization methods to prevent overfitting, namely early stopping and weight decay. Early stopping was achieved by stopping the training when there was no improvement in two subsequent iterations on the validation set. As weight decay regularization the weights were scaled back after each iteration, forcing them to converge to smaller absolute values than they otherwise would. The neural networks were trained using semi-batch backpropagation, the batch size being 100. The initial learn rate was set to and held fixed while the error on the development set kept decreasing. Afterwards, if the error rate did not decrease in the given iteration, then the learn rate was subsequently halved. We used our custom implementation for neural networks, which was implemented to run on GPU [11]. The training of a DRN on the synthetic dataset took less than 8 minutes using an NVIDIA GTX-770 graphics card Input Data and Block Size For the input data, various options were available and a separate training was performed with each type. The first choice was to train a neural network on raw pixel data, read in the discussed pattern for each block. Binary version of the vectors was also evaluated to compare efficiency of the neural networks on grayscale and binary images. Furthermore, since QR code has a well-defined, strict structure, it means blocks of QR code parts probably have very specific components in the frequency domain. This assumption motivated us to perform experiments with vectors in that domain. We transformed the training vectors to the DCT and DFT domains, and evaluated neural networks on both sets. Furthermore, a final neural network has been trained to the edge map, since the structure of QR codes also suggest very specific edge layouts that probably can be fed to a neural network and result in efficient training. Training vectors in that case consisted of pixels of the unified magnitude map of Sobel-X and Sobel-Y gradients (Fig. 2), read in circular pattern. For making a decision about block size, two conditions have to be fulfilled. Block size should be small enough to having many of them to build a QR code, so threshold would be chosen easily for dropping or keeping clusters in the probability matrix. On the other hand, block size has to be large enough to provide prominent characteristics in the input vec-

3 (a) example synthetic image (b) edge magnitude map (a) σ = 0.07, α = 0.39 (b) σ = 0.65, α = 0.77 (c) σ = 1.32, α = 0.05 (d) σ = 1.88, α = 0.95 (e) σ = 2.86, α = 0.65 (f) σ = 2.99, α = 0.69 Fig. 2. Example synthetic image and Sobel magnitude image. Edge maps seem to be reliable input for the neural network training, and it is reassured in the results section tors of the neural network. Papers on visual code localization suggest empirically [12], or using geometrical approach [10], that optimal tile size is about 1/3 of the smaller dimension of the examined visual code in case of features based on image partitioning. This choice for block size was our initial value, however, we evaluated neural networks with input vectors of different block sizes. After making a decision about a suitable block size, the amount of overlap also has to be decided. Different offsets for blocks having the optimal size are also evaluated empirically, because papers on this topic do not categorically declare if overlapping increases localization performance. 3. EVALUATION AND RESULTS The test database consists of synthetic and 100 arbitrarily acquired images containing QR code. The synthetic examples are built with a computer-generated QR code containing all of the lower- and uppercase letters of the alphabet in random order. This QR code was placed on a random negative image, with perspective transformation. After that, Gaussian smoothing and noise have been gradually added to the images, ranging [0,3] for the σ of the Gaussian kernel. For noise, a noise image (In ) was generated with intensities ranging from [-127, 127] following normal distribution, and added gradually to the original 8-bit image (Io ) as I = αin + (1 α)io, with α ranging [0, 0.5]. Some samples with parameters being in the discussed ranges are present in (Fig. 3). A total number of 1.8 million vectors were extracted from those images, about of them are labeled as positive. Real images were taken with a 3.2 Mpx Huawei handheld phone camera. Significant smoothing is present on those images due to the absence of flash and precise focusing capability. These images contain a printed QR code that mostly suffer from minor bending, placed on various textured backgrounds, like creased cloth, carpet, marble or cover page of a book (Fig. 1(a) and 4(d)), in order to create high background variability and a make the classification task more complex. Fig. 3. Samples of the training database with different amount of smoothing and noise. The arbitrarily acquired image set roughly had QR codes of the same size as the artificial set, and its extracted vector set consisted about vectors, about of them being positive. Images were converted to grayscale before the vector extraction Input Types The first group of neural network training was performed using different input data type and range as well, in order to determine the best available option for input vectors. This first test set did not contain full images, only the vectors of blocks that has 100 % coverage of QR code part, and an equal number of random negative blocks, about vectors in total. Both normal and deep rectifier neural networks have been trained for the data, as shown in Table 1. For this first test, only two portions of the database were separated, for

4 Input data Input range Precision Hit rate F-measure ANN Raw pixels 8-bit Raw pixels binary DCT 8-bit DCT binary DFT 8-bit DFT binary Sobel-XY 8-bit DRN Raw pixels 8-bit Raw pixels binary DCT 8-bit DCT binary DFT 8-bit DFT binary Sobel-XY 8-bit Table 1. Training results for different input data types and ranges. NN type Precision Hit rate F-measure ANN DRN Table 2. Evaluation of the neural networks on arbitrarily acquired images. Input vectors were 8-bit raw pixels. Inferior DRN precision indicates that these neural nets need more training samples compared to ANNs. training and testing purposes in 8:2 proportion. For later tests, the smaller set has been further divided into test and validation parts. Results show that deep neural networks outperform regular ones in general, and 8-bit input data facilitates more accurate training. Application of DCT and DFT to the vectors do not influence the training significantly, and even though those transformations are easy to compute on GPUs, they do not seem to be worth the computational cost. However, vectors of Sobel magnitude images slightly improve training efficiency, which could be expected due to the QR code edge structure (Fig. 2). Neural networks of this first evaluation used px block size, which is roughly 1/3 of the expected QR code size. Block offset for overlapping blocks was set to 10 px, since element size of the generated QR code was also that large. Each vector consisted of 128 points sampled from the circular pattern within the block. After training these neural networks on the synthetic database, neural networks that were trained on the raw, 8-bit pixel data, have been evaluated on the arbitrarily acquired image set. Precision dropped to about 0.4, but hit rate is still acceptable. Both shallow and deep rectifier neural networks have been evaluated, as shown in Table 2. The precision drop is probably due to the various textures, like tablecloth, wood and marble, present in the real images. NN type Data type Precision Hit rate F-measure T + = 0.1 ANN Real DRN Real ANN Synthetic DRN Synthetic T + = 0.5 ANN Real DRN Real ANN Synthetic DRN Synthetic Table 3. Evaluation of the neural networks also having vectors of partially covered blocks as input, with different thresholds for input labels Subset Opt. Thresh. max(f 1) AUC All vectors N + N N + b N b Table 4. DRN results on different subsets of input vectors of synthetic images. N + and N denote for positive and negative samples, while N + b and N b are subsets of N + and N with partially covered blocks excluded 3.2. Partially Covered Blocks In the first case, training was only performed using vectors of background and blocks fully covered with QR code parts. As the next step of this experiment, the input data has been extended for partially covered blocks, thresholded at 0.1 and 0.5 coverage ratios for positive labeling. No filtering has been applied to the amount of negatively labeled vectors. Both ANNs and DRNs were trained on vectors of both synthetic and real images, separately. The database has been divided to 8:1:1 ratio for training, testing and validation parts, respectively. Results (Table 3) show that allowing the partially covered samples for the training significantly drops the precision of the NNs. Furthermore, threshold of 0.1 for the positive labels keeps the hit rate in a satisfactory level, while 0.5 decreases it largely (Fig. 4). Still, the low hit rate is not a problem, since enough positively predicted blocks remain in the probability matrix for being able to indicate a QR code candidate even in those cases (Fig. 4(e)). It can be concluded that NNs trained on full images and on a reduced subset of vectors, where positively and negatively labeled samples are roughly in the same amount, do not differ significantly, while excluding vectors that come from partially covered blocks drops the performance noticeably. Table 4 summarizes results measured on the discussed training sets.

5 max(f) (a) max(f ) (a) Synthetic image (b) ANN feature image (c) DRN feature image (d) Real image AUC (e) ANN feature image (f) DRN feature image Fig. 4. Original and feature images built from the output of the neural networks. Even though ANNs have weaker classification power and miss several blocks, they can indicate the presence of QR code candidates strongly enough Parameters and Run Times We used our custom implementation for neural networks, which was implemented to run on GPU. The training of a DRN on the synthetic dataset took less than 8 minutes using an NVIDIA GTX-770 grapics card. The computation power of this setup allowed to process about vectors per second, which means real-time processing of px images with cca. 5 FPS using px for block size and 10 px for offset. The QR codes the evaluation has been performed on, had 10 px element size. Block sizes of the input vectors from px to 90 px have been evaluated. Block sizes smaller than px would lead the training to learn solid black and white blocks as positive, thus heavily raising false positive rate. Larger than 90 px block sizes would also decrease performance, since large block size drastically decreases the number of positive samples available for the training while introducing a vast amount of partially covered blocks that are harder to train onto. The numbers of Fig. 5 also show that the proposed method is robust for expected QR code size, since block size range for good performance is large enough according to code dimensions. Fig. 6 shows results for DRNs using different rate of overlapping, up to px (no overlap). It is notable that F-score only slightly differs considering overlap from 10 to px, while the amount of input vectors is drastically smaller. After all these experiments, a DRN has been trained with the best experimental training setup, in order to compare it with other localization methods of the literature. According to Table 5, NN training is proven to be a viable option for real-time efficient QR code localization. As the first approach to compare, an algorithm based on mathematical morphology has been selected [2], since it is also a general purpose toolset (b) AUC Fig. 5. DRN results respect to input block size max(f) (a) max(f ) AUC (b) AUC Fig. 6. DRN results respect to input block offset, using px block size. of image processing, as NNs are for machine learning tasks. This reference method is reliable, but due to the morphological operations, it requires the most computational capacity and gives the slowest processing speed, from 900 to 10 ms per image. Another work in QR code localization is using a specific, image-based classifier training. It proposes training a cascase of classifiers using Haar-like features [3]. 4. CONCLUDING REMARKS Usage of neural networks is a robust, reliable approach for fast QR code localization. We have examined performance of shallow and deep rectifier neural networks for the task as well, on various input types. We have proven efficiency of NNs on a large amount of input images, both synthetic and real. It is observed that DRNs perform better than ANNs in general. The best choice for input data is the edge magnitude map. Block size does not interfere with DRN accuracy signif-

6 Algorithm Precision Recall F-measure Proposed REF-HAAR [3] REF-MORPH [2] Table 5. Performance comparison of the best NN to other algorithms icantly, and has a wide working range between 10 to percent of the expected QR code dimensions. Even with dense overlapping at blocks, real-time image processing is possible. Our future work includes experimenting with other patterns and features for the neural network training. Acknowledgement This publication is supported by the European Union and co-funded by the European Social Fund. Project title: Telemedicine-oriented research activities in the fields of mathematics, informatics and medical sciences. Project number: TÁMOP A-11/1/KONV REFERENCES [1] Chung-Hua Chu, De-Nian Yang, Ya-Lan Pan, and Ming-Syan Chen, Stabilization and extraction of 2D barcodes for camera phones, Multimedia Systems, pp [7] Geoffrey E Hinton, Simon Osindero, and Yee-Whye Teh, A fast learning algorithm for deep belief nets, Neural Computation, vol. 18, no. 7, pp , [8] Frank Seide, Gang Li, Xie Chen, and Dong Yu, Feature engineering in context-dependent deep neural networks for conversational speech transcription, in Proc. ASRU, 2011, pp [9] Xavier Glorot, Antoine Bordes, and Yoshua Bengio, Deep sparse rectifier networks, in Proc. AISTATS, 2011, pp [10] Péter Bodnár and László G. Nyúl, A novel method for barcode localization in image domain, in Image Analysis and Recognition. 2013, vol. 79 of Lecture Notes in Computer Science, pp , Springer Berlin Heidelberg. [11] László Tóth and Tamás Grósz, A comparison of deep neural network training methods for large vocabulary speech recognition, in Proceedings of TSD, 2013, pp [12] Péter Bodnár and László G. Nyúl, Improving barcode detection with combination of simple detectors, in The 8th International Conference on Signal Image Technology (SITIS 2012), 2012, pp [2] Eisaku Ohbuchi, Hiroshi Hanaizumi, and Lim Ah Hock, Barcode readers using the camera device in mobile phones, in Cyberworlds, 2004 International Conference on, 2004, pp [3] Luiz F. F. Belussi and Nina S. T. Hirata, Fast QR code detection in arbitrarily acquired images, in Graphics, Patterns and Images (Sibgrapi), th SIBGRAPI Conference on, 2011, pp [4] Gábor Sörös and Christian Flörkemeier, Blur-resistant joint 1D and 2D barcode localization for smartphones, in Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia, New York, NY, USA, 2013, MUM 13, pp. 11:1 11:8, ACM. [5] István Szentandrási, Adam Herout, and Markéta Dubská, Fast detection and recognition of QR codes in high-resolution images, in Proceedings of the 28th Spring Conference on Computer Graphics, New York, NY, USA, 2013, SCCG 12, pp , ACM. [6] Xavier Glorot and Yoshua Bengio, Understanding the difficulty of training deep feedforward neural networks, in Proc. AISTATS, 2010, pp

Improved QR Code Localization Using Boosted Cascade of Weak Classifiers

Improved QR Code Localization Using Boosted Cascade of Weak Classifiers Acta Cybernetica 22 (2015) 21 33. Improved QR Code Localization Using Boosted Cascade of Weak Classifiers Péter Bodnár and László G. Nyúl Abstract Usage of computer-readable visual codes became common

More information

Deep Learning with Tensorflow AlexNet

Deep Learning with Tensorflow   AlexNet Machine Learning and Computer Vision Group Deep Learning with Tensorflow http://cvml.ist.ac.at/courses/dlwt_w17/ AlexNet Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton, "Imagenet classification

More information

Comparing Dropout Nets to Sum-Product Networks for Predicting Molecular Activity

Comparing Dropout Nets to Sum-Product Networks for Predicting Molecular Activity 000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050

More information

Neural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani

Neural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer

More information

Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah

Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah Reference Most of the slides are taken from the third chapter of the online book by Michael Nielson: neuralnetworksanddeeplearning.com

More information

COMP9444 Neural Networks and Deep Learning 7. Image Processing. COMP9444 c Alan Blair, 2017

COMP9444 Neural Networks and Deep Learning 7. Image Processing. COMP9444 c Alan Blair, 2017 COMP9444 Neural Networks and Deep Learning 7. Image Processing COMP9444 17s2 Image Processing 1 Outline Image Datasets and Tasks Convolution in Detail AlexNet Weight Initialization Batch Normalization

More information

Traffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers

Traffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers Traffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers A. Salhi, B. Minaoui, M. Fakir, H. Chakib, H. Grimech Faculty of science and Technology Sultan Moulay Slimane

More information

COMP9444 Neural Networks and Deep Learning 5. Geometry of Hidden Units

COMP9444 Neural Networks and Deep Learning 5. Geometry of Hidden Units COMP9 8s Geometry of Hidden Units COMP9 Neural Networks and Deep Learning 5. Geometry of Hidden Units Outline Geometry of Hidden Unit Activations Limitations of -layer networks Alternative transfer functions

More information

Lecture 13. Deep Belief Networks. Michael Picheny, Bhuvana Ramabhadran, Stanley F. Chen

Lecture 13. Deep Belief Networks. Michael Picheny, Bhuvana Ramabhadran, Stanley F. Chen Lecture 13 Deep Belief Networks Michael Picheny, Bhuvana Ramabhadran, Stanley F. Chen IBM T.J. Watson Research Center Yorktown Heights, New York, USA {picheny,bhuvana,stanchen}@us.ibm.com 12 December 2012

More information

Neural Network Optimization and Tuning / Spring 2018 / Recitation 3

Neural Network Optimization and Tuning / Spring 2018 / Recitation 3 Neural Network Optimization and Tuning 11-785 / Spring 2018 / Recitation 3 1 Logistics You will work through a Jupyter notebook that contains sample and starter code with explanations and comments throughout.

More information

CS489/698: Intro to ML

CS489/698: Intro to ML CS489/698: Intro to ML Lecture 14: Training of Deep NNs Instructor: Sun Sun 1 Outline Activation functions Regularization Gradient-based optimization 2 Examples of activation functions 3 5/28/18 Sun Sun

More information

COMP 551 Applied Machine Learning Lecture 16: Deep Learning

COMP 551 Applied Machine Learning Lecture 16: Deep Learning COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all

More information

Deep Learning. Deep Learning provided breakthrough results in speech recognition and image classification. Why?

Deep Learning. Deep Learning provided breakthrough results in speech recognition and image classification. Why? Data Mining Deep Learning Deep Learning provided breakthrough results in speech recognition and image classification. Why? Because Speech recognition and image classification are two basic examples of

More information

Chapter 2 Basic Structure of High-Dimensional Spaces

Chapter 2 Basic Structure of High-Dimensional Spaces Chapter 2 Basic Structure of High-Dimensional Spaces Data is naturally represented geometrically by associating each record with a point in the space spanned by the attributes. This idea, although simple,

More information

Natural Language Processing CS 6320 Lecture 6 Neural Language Models. Instructor: Sanda Harabagiu

Natural Language Processing CS 6320 Lecture 6 Neural Language Models. Instructor: Sanda Harabagiu Natural Language Processing CS 6320 Lecture 6 Neural Language Models Instructor: Sanda Harabagiu In this lecture We shall cover: Deep Neural Models for Natural Language Processing Introduce Feed Forward

More information

Neural Networks: promises of current research

Neural Networks: promises of current research April 2008 www.apstat.com Current research on deep architectures A few labs are currently researching deep neural network training: Geoffrey Hinton s lab at U.Toronto Yann LeCun s lab at NYU Our LISA lab

More information

CPSC 340: Machine Learning and Data Mining. Deep Learning Fall 2016

CPSC 340: Machine Learning and Data Mining. Deep Learning Fall 2016 CPSC 340: Machine Learning and Data Mining Deep Learning Fall 2016 Assignment 5: Due Friday. Assignment 6: Due next Friday. Final: Admin December 12 (8:30am HEBB 100) Covers Assignments 1-6. Final from

More information

Neural Network and Deep Learning. Donglin Zeng, Department of Biostatistics, University of North Carolina

Neural Network and Deep Learning. Donglin Zeng, Department of Biostatistics, University of North Carolina Neural Network and Deep Learning Early history of deep learning Deep learning dates back to 1940s: known as cybernetics in the 1940s-60s, connectionism in the 1980s-90s, and under the current name starting

More information

ORT EP R RCH A ESE R P A IDI! " #$$% &' (# $!"

ORT EP R RCH A ESE R P A IDI!  #$$% &' (# $! R E S E A R C H R E P O R T IDIAP A Parallel Mixture of SVMs for Very Large Scale Problems Ronan Collobert a b Yoshua Bengio b IDIAP RR 01-12 April 26, 2002 Samy Bengio a published in Neural Computation,

More information

Robustness of Selective Desensitization Perceptron Against Irrelevant and Partially Relevant Features in Pattern Classification

Robustness of Selective Desensitization Perceptron Against Irrelevant and Partially Relevant Features in Pattern Classification Robustness of Selective Desensitization Perceptron Against Irrelevant and Partially Relevant Features in Pattern Classification Tomohiro Tanno, Kazumasa Horie, Jun Izawa, and Masahiko Morita University

More information

Convolutional Layer Pooling Layer Fully Connected Layer Regularization

Convolutional Layer Pooling Layer Fully Connected Layer Regularization Semi-Parallel Deep Neural Networks (SPDNN), Convergence and Generalization Shabab Bazrafkan, Peter Corcoran Center for Cognitive, Connected & Computational Imaging, College of Engineering & Informatics,

More information

DEEP LEARNING REVIEW. Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature Presented by Divya Chitimalla

DEEP LEARNING REVIEW. Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature Presented by Divya Chitimalla DEEP LEARNING REVIEW Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature 2015 -Presented by Divya Chitimalla What is deep learning Deep learning allows computational models that are composed of multiple

More information

Dynamic Routing Between Capsules

Dynamic Routing Between Capsules Report Explainable Machine Learning Dynamic Routing Between Capsules Author: Michael Dorkenwald Supervisor: Dr. Ullrich Köthe 28. Juni 2018 Inhaltsverzeichnis 1 Introduction 2 2 Motivation 2 3 CapusleNet

More information

Advanced Introduction to Machine Learning, CMU-10715

Advanced Introduction to Machine Learning, CMU-10715 Advanced Introduction to Machine Learning, CMU-10715 Deep Learning Barnabás Póczos, Sept 17 Credits Many of the pictures, results, and other materials are taken from: Ruslan Salakhutdinov Joshua Bengio

More information

Deep Learning. Volker Tresp Summer 2014

Deep Learning. Volker Tresp Summer 2014 Deep Learning Volker Tresp Summer 2014 1 Neural Network Winter and Revival While Machine Learning was flourishing, there was a Neural Network winter (late 1990 s until late 2000 s) Around 2010 there

More information

Practical Tips for using Backpropagation

Practical Tips for using Backpropagation Practical Tips for using Backpropagation Keith L. Downing August 31, 2017 1 Introduction In practice, backpropagation is as much an art as a science. The user typically needs to try many combinations of

More information

4.12 Generalization. In back-propagation learning, as many training examples as possible are typically used.

4.12 Generalization. In back-propagation learning, as many training examples as possible are typically used. 1 4.12 Generalization In back-propagation learning, as many training examples as possible are typically used. It is hoped that the network so designed generalizes well. A network generalizes well when

More information

INTELLIGENT transportation systems have a significant

INTELLIGENT transportation systems have a significant INTL JOURNAL OF ELECTRONICS AND TELECOMMUNICATIONS, 205, VOL. 6, NO. 4, PP. 35 356 Manuscript received October 4, 205; revised November, 205. DOI: 0.55/eletel-205-0046 Efficient Two-Step Approach for Automatic

More information

A modified and fast Perceptron learning rule and its use for Tag Recommendations in Social Bookmarking Systems

A modified and fast Perceptron learning rule and its use for Tag Recommendations in Social Bookmarking Systems A modified and fast Perceptron learning rule and its use for Tag Recommendations in Social Bookmarking Systems Anestis Gkanogiannis and Theodore Kalamboukis Department of Informatics Athens University

More information

CS231A Course Project Final Report Sign Language Recognition with Unsupervised Feature Learning

CS231A Course Project Final Report Sign Language Recognition with Unsupervised Feature Learning CS231A Course Project Final Report Sign Language Recognition with Unsupervised Feature Learning Justin Chen Stanford University justinkchen@stanford.edu Abstract This paper focuses on experimenting with

More information

JOINT INTENT DETECTION AND SLOT FILLING USING CONVOLUTIONAL NEURAL NETWORKS. Puyang Xu, Ruhi Sarikaya. Microsoft Corporation

JOINT INTENT DETECTION AND SLOT FILLING USING CONVOLUTIONAL NEURAL NETWORKS. Puyang Xu, Ruhi Sarikaya. Microsoft Corporation JOINT INTENT DETECTION AND SLOT FILLING USING CONVOLUTIONAL NEURAL NETWORKS Puyang Xu, Ruhi Sarikaya Microsoft Corporation ABSTRACT We describe a joint model for intent detection and slot filling based

More information

Classification of objects from Video Data (Group 30)

Classification of objects from Video Data (Group 30) Classification of objects from Video Data (Group 30) Sheallika Singh 12665 Vibhuti Mahajan 12792 Aahitagni Mukherjee 12001 M Arvind 12385 1 Motivation Video surveillance has been employed for a long time

More information

SEMANTIC COMPUTING. Lecture 8: Introduction to Deep Learning. TU Dresden, 7 December Dagmar Gromann International Center For Computational Logic

SEMANTIC COMPUTING. Lecture 8: Introduction to Deep Learning. TU Dresden, 7 December Dagmar Gromann International Center For Computational Logic SEMANTIC COMPUTING Lecture 8: Introduction to Deep Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 7 December 2018 Overview Introduction Deep Learning General Neural Networks

More information

Visual object classification by sparse convolutional neural networks

Visual object classification by sparse convolutional neural networks Visual object classification by sparse convolutional neural networks Alexander Gepperth 1 1- Ruhr-Universität Bochum - Institute for Neural Dynamics Universitätsstraße 150, 44801 Bochum - Germany Abstract.

More information

Image Compression: An Artificial Neural Network Approach

Image Compression: An Artificial Neural Network Approach Image Compression: An Artificial Neural Network Approach Anjana B 1, Mrs Shreeja R 2 1 Department of Computer Science and Engineering, Calicut University, Kuttippuram 2 Department of Computer Science and

More information

Neuron Selectivity as a Biologically Plausible Alternative to Backpropagation

Neuron Selectivity as a Biologically Plausible Alternative to Backpropagation Neuron Selectivity as a Biologically Plausible Alternative to Backpropagation C.J. Norsigian Department of Bioengineering cnorsigi@eng.ucsd.edu Vishwajith Ramesh Department of Bioengineering vramesh@eng.ucsd.edu

More information

Deep Learning for Computer Vision

Deep Learning for Computer Vision Deep Learning for Computer Vision Lecture 7: Universal Approximation Theorem, More Hidden Units, Multi-Class Classifiers, Softmax, and Regularization Peter Belhumeur Computer Science Columbia University

More information

Character Recognition Using Convolutional Neural Networks

Character Recognition Using Convolutional Neural Networks Character Recognition Using Convolutional Neural Networks David Bouchain Seminar Statistical Learning Theory University of Ulm, Germany Institute for Neural Information Processing Winter 2006/2007 Abstract

More information

Research on Pruning Convolutional Neural Network, Autoencoder and Capsule Network

Research on Pruning Convolutional Neural Network, Autoencoder and Capsule Network Research on Pruning Convolutional Neural Network, Autoencoder and Capsule Network Tianyu Wang Australia National University, Colledge of Engineering and Computer Science u@anu.edu.au Abstract. Some tasks,

More information

Locating 1-D Bar Codes in DCT-Domain

Locating 1-D Bar Codes in DCT-Domain Edith Cowan University Research Online ECU Publications Pre. 2011 2006 Locating 1-D Bar Codes in DCT-Domain Alexander Tropf Edith Cowan University Douglas Chai Edith Cowan University 10.1109/ICASSP.2006.1660449

More information

Robust PDF Table Locator

Robust PDF Table Locator Robust PDF Table Locator December 17, 2016 1 Introduction Data scientists rely on an abundance of tabular data stored in easy-to-machine-read formats like.csv files. Unfortunately, most government records

More information

Digital Image Processing (CS/ECE 545) Lecture 5: Edge Detection (Part 2) & Corner Detection

Digital Image Processing (CS/ECE 545) Lecture 5: Edge Detection (Part 2) & Corner Detection Digital Image Processing (CS/ECE 545) Lecture 5: Edge Detection (Part 2) & Corner Detection Prof Emmanuel Agu Computer Science Dept. Worcester Polytechnic Institute (WPI) Recall: Edge Detection Image processing

More information

Stacked Denoising Autoencoders for Face Pose Normalization

Stacked Denoising Autoencoders for Face Pose Normalization Stacked Denoising Autoencoders for Face Pose Normalization Yoonseop Kang 1, Kang-Tae Lee 2,JihyunEun 2, Sung Eun Park 2 and Seungjin Choi 1 1 Department of Computer Science and Engineering Pohang University

More information

Short Survey on Static Hand Gesture Recognition

Short Survey on Static Hand Gesture Recognition Short Survey on Static Hand Gesture Recognition Huu-Hung Huynh University of Science and Technology The University of Danang, Vietnam Duc-Hoang Vo University of Science and Technology The University of

More information

CS6220: DATA MINING TECHNIQUES

CS6220: DATA MINING TECHNIQUES CS6220: DATA MINING TECHNIQUES Image Data: Classification via Neural Networks Instructor: Yizhou Sun yzsun@ccs.neu.edu November 19, 2015 Methods to Learn Classification Clustering Frequent Pattern Mining

More information

Deep Learning. Architecture Design for. Sargur N. Srihari

Deep Learning. Architecture Design for. Sargur N. Srihari Architecture Design for Deep Learning Sargur N. srihari@cedar.buffalo.edu 1 Topics Overview 1. Example: Learning XOR 2. Gradient-Based Learning 3. Hidden Units 4. Architecture Design 5. Backpropagation

More information

Machine Learning. The Breadth of ML Neural Networks & Deep Learning. Marc Toussaint. Duy Nguyen-Tuong. University of Stuttgart

Machine Learning. The Breadth of ML Neural Networks & Deep Learning. Marc Toussaint. Duy Nguyen-Tuong. University of Stuttgart Machine Learning The Breadth of ML Neural Networks & Deep Learning Marc Toussaint University of Stuttgart Duy Nguyen-Tuong Bosch Center for Artificial Intelligence Summer 2017 Neural Networks Consider

More information

An Empirical Evaluation of Deep Architectures on Problems with Many Factors of Variation

An Empirical Evaluation of Deep Architectures on Problems with Many Factors of Variation An Empirical Evaluation of Deep Architectures on Problems with Many Factors of Variation Hugo Larochelle, Dumitru Erhan, Aaron Courville, James Bergstra, and Yoshua Bengio Université de Montréal 13/06/2007

More information

Efficient 1D and 2D barcode detection using mathematical morphology

Efficient 1D and 2D barcode detection using mathematical morphology Efficient 1D and 2D barcode detection using mathematical morphology Melinda Katona and László G. Nyúl Department of Image Processing and Computer Graphics University of Szeged Árpád tér 2, H-6720 Szeged,

More information

Link Prediction for Social Network

Link Prediction for Social Network Link Prediction for Social Network Ning Lin Computer Science and Engineering University of California, San Diego Email: nil016@eng.ucsd.edu Abstract Friendship recommendation has become an important issue

More information

An ICA based Approach for Complex Color Scene Text Binarization

An ICA based Approach for Complex Color Scene Text Binarization An ICA based Approach for Complex Color Scene Text Binarization Siddharth Kherada IIIT-Hyderabad, India siddharth.kherada@research.iiit.ac.in Anoop M. Namboodiri IIIT-Hyderabad, India anoop@iiit.ac.in

More information

I. INTRODUCTION. Figure-1 Basic block of text analysis

I. INTRODUCTION. Figure-1 Basic block of text analysis ISSN: 2349-7637 (Online) (RHIMRJ) Research Paper Available online at: www.rhimrj.com Detection and Localization of Texts from Natural Scene Images: A Hybrid Approach Priyanka Muchhadiya Post Graduate Fellow,

More information

Convolutional Neural Networks

Convolutional Neural Networks Lecturer: Barnabas Poczos Introduction to Machine Learning (Lecture Notes) Convolutional Neural Networks Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.

More information

A novel supervised learning algorithm and its use for Spam Detection in Social Bookmarking Systems

A novel supervised learning algorithm and its use for Spam Detection in Social Bookmarking Systems A novel supervised learning algorithm and its use for Spam Detection in Social Bookmarking Systems Anestis Gkanogiannis and Theodore Kalamboukis Department of Informatics Athens University of Economics

More information

Akarsh Pokkunuru EECS Department Contractive Auto-Encoders: Explicit Invariance During Feature Extraction

Akarsh Pokkunuru EECS Department Contractive Auto-Encoders: Explicit Invariance During Feature Extraction Akarsh Pokkunuru EECS Department 03-16-2017 Contractive Auto-Encoders: Explicit Invariance During Feature Extraction 1 AGENDA Introduction to Auto-encoders Types of Auto-encoders Analysis of different

More information

Deep Learning Cook Book

Deep Learning Cook Book Deep Learning Cook Book Robert Haschke (CITEC) Overview Input Representation Output Layer + Cost Function Hidden Layer Units Initialization Regularization Input representation Choose an input representation

More information

2. Neural network basics

2. Neural network basics 2. Neural network basics Next commonalities among different neural networks are discussed in order to get started and show which structural parts or concepts appear in almost all networks. It is presented

More information

Application of Support Vector Machine Algorithm in Spam Filtering

Application of Support Vector Machine Algorithm in  Spam Filtering Application of Support Vector Machine Algorithm in E-Mail Spam Filtering Julia Bluszcz, Daria Fitisova, Alexander Hamann, Alexey Trifonov, Advisor: Patrick Jähnichen Abstract The problem of spam classification

More information

Object Detection Lecture Introduction to deep learning (CNN) Idar Dyrdal

Object Detection Lecture Introduction to deep learning (CNN) Idar Dyrdal Object Detection Lecture 10.3 - Introduction to deep learning (CNN) Idar Dyrdal Deep Learning Labels Computational models composed of multiple processing layers (non-linear transformations) Used to learn

More information

ECG782: Multidimensional Digital Signal Processing

ECG782: Multidimensional Digital Signal Processing ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting

More information

DEEP LEARNING OF COMPRESSED SENSING OPERATORS WITH STRUCTURAL SIMILARITY (SSIM) LOSS

DEEP LEARNING OF COMPRESSED SENSING OPERATORS WITH STRUCTURAL SIMILARITY (SSIM) LOSS DEEP LEARNING OF COMPRESSED SENSING OPERATORS WITH STRUCTURAL SIMILARITY (SSIM) LOSS ABSTRACT Compressed sensing (CS) is a signal processing framework for efficiently reconstructing a signal from a small

More information

AIIA shot boundary detection at TRECVID 2006

AIIA shot boundary detection at TRECVID 2006 AIIA shot boundary detection at TRECVID 6 Z. Černeková, N. Nikolaidis and I. Pitas Artificial Intelligence and Information Analysis Laboratory Department of Informatics Aristotle University of Thessaloniki

More information

Combined Weak Classifiers

Combined Weak Classifiers Combined Weak Classifiers Chuanyi Ji and Sheng Ma Department of Electrical, Computer and System Engineering Rensselaer Polytechnic Institute, Troy, NY 12180 chuanyi@ecse.rpi.edu, shengm@ecse.rpi.edu Abstract

More information

Fabric Defect Detection Based on Computer Vision

Fabric Defect Detection Based on Computer Vision Fabric Defect Detection Based on Computer Vision Jing Sun and Zhiyu Zhou College of Information and Electronics, Zhejiang Sci-Tech University, Hangzhou, China {jings531,zhouzhiyu1993}@163.com Abstract.

More information

A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images

A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images Marc Aurelio Ranzato Yann LeCun Courant Institute of Mathematical Sciences New York University - New York, NY 10003 Abstract

More information

Neural Networks for Machine Learning. Lecture 15a From Principal Components Analysis to Autoencoders

Neural Networks for Machine Learning. Lecture 15a From Principal Components Analysis to Autoencoders Neural Networks for Machine Learning Lecture 15a From Principal Components Analysis to Autoencoders Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed Principal Components

More information

Edge Detection for Dental X-ray Image Segmentation using Neural Network approach

Edge Detection for Dental X-ray Image Segmentation using Neural Network approach Volume 1, No. 7, September 2012 ISSN 2278-1080 The International Journal of Computer Science & Applications (TIJCSA) RESEARCH PAPER Available Online at http://www.journalofcomputerscience.com/ Edge Detection

More information

CS 4510/9010 Applied Machine Learning

CS 4510/9010 Applied Machine Learning CS 4510/9010 Applied Machine Learning Neural Nets Paula Matuszek Spring, 2015 1 Neural Nets, the very short version A neural net consists of layers of nodes, or neurons, each of which has an activation

More information

Accelerometer Gesture Recognition

Accelerometer Gesture Recognition Accelerometer Gesture Recognition Michael Xie xie@cs.stanford.edu David Pan napdivad@stanford.edu December 12, 2014 Abstract Our goal is to make gesture-based input for smartphones and smartwatches accurate

More information

Facial Expression Classification with Random Filters Feature Extraction

Facial Expression Classification with Random Filters Feature Extraction Facial Expression Classification with Random Filters Feature Extraction Mengye Ren Facial Monkey mren@cs.toronto.edu Zhi Hao Luo It s Me lzh@cs.toronto.edu I. ABSTRACT In our work, we attempted to tackle

More information

Facial Keypoint Detection

Facial Keypoint Detection Facial Keypoint Detection CS365 Artificial Intelligence Abheet Aggarwal 12012 Ajay Sharma 12055 Abstract Recognizing faces is a very challenging problem in the field of image processing. The techniques

More information

A Laplacian Based Novel Approach to Efficient Text Localization in Grayscale Images

A Laplacian Based Novel Approach to Efficient Text Localization in Grayscale Images A Laplacian Based Novel Approach to Efficient Text Localization in Grayscale Images Karthik Ram K.V & Mahantesh K Department of Electronics and Communication Engineering, SJB Institute of Technology, Bangalore,

More information

Introduction to Deep Learning

Introduction to Deep Learning ENEE698A : Machine Learning Seminar Introduction to Deep Learning Raviteja Vemulapalli Image credit: [LeCun 1998] Resources Unsupervised feature learning and deep learning (UFLDL) tutorial (http://ufldl.stanford.edu/wiki/index.php/ufldl_tutorial)

More information

Deep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks

Deep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks Deep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks Si Chen The George Washington University sichen@gwmail.gwu.edu Meera Hahn Emory University mhahn7@emory.edu Mentor: Afshin

More information

Learning to Match. Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li

Learning to Match. Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li Learning to Match Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li 1. Introduction The main tasks in many applications can be formalized as matching between heterogeneous objects, including search, recommendation,

More information

Traffic Sign Localization and Classification Methods: An Overview

Traffic Sign Localization and Classification Methods: An Overview Traffic Sign Localization and Classification Methods: An Overview Ivan Filković University of Zagreb Faculty of Electrical Engineering and Computing Department of Electronics, Microelectronics, Computer

More information

Autoencoders, denoising autoencoders, and learning deep networks

Autoencoders, denoising autoencoders, and learning deep networks 4 th CiFAR Summer School on Learning and Vision in Biology and Engineering Toronto, August 5-9 2008 Autoencoders, denoising autoencoders, and learning deep networks Part II joint work with Hugo Larochelle,

More information

Perceptron: This is convolution!

Perceptron: This is convolution! Perceptron: This is convolution! v v v Shared weights v Filter = local perceptron. Also called kernel. By pooling responses at different locations, we gain robustness to the exact spatial location of image

More information

Apparel Classifier and Recommender using Deep Learning

Apparel Classifier and Recommender using Deep Learning Apparel Classifier and Recommender using Deep Learning Live Demo at: http://saurabhg.me/projects/tag-that-apparel Saurabh Gupta sag043@ucsd.edu Siddhartha Agarwal siagarwa@ucsd.edu Apoorve Dave a1dave@ucsd.edu

More information

Dropout. Sargur N. Srihari This is part of lecture slides on Deep Learning:

Dropout. Sargur N. Srihari This is part of lecture slides on Deep Learning: Dropout Sargur N. srihari@buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Regularization Strategies 1. Parameter Norm Penalties 2. Norm Penalties

More information

Transfer Learning Using Rotated Image Data to Improve Deep Neural Network Performance

Transfer Learning Using Rotated Image Data to Improve Deep Neural Network Performance Transfer Learning Using Rotated Image Data to Improve Deep Neural Network Performance Telmo Amaral¹, Luís M. Silva¹², Luís A. Alexandre³, Chetak Kandaswamy¹, Joaquim Marques de Sá¹ 4, and Jorge M. Santos¹

More information

Face Detection using Hierarchical SVM

Face Detection using Hierarchical SVM Face Detection using Hierarchical SVM ECE 795 Pattern Recognition Christos Kyrkou Fall Semester 2010 1. Introduction Face detection in video is the process of detecting and classifying small images extracted

More information

One Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models

One Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models One Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models [Supplemental Materials] 1. Network Architecture b ref b ref +1 We now describe the architecture of the networks

More information

Deep Learning Basic Lecture - Complex Systems & Artificial Intelligence 2017/18 (VO) Asan Agibetov, PhD.

Deep Learning Basic Lecture - Complex Systems & Artificial Intelligence 2017/18 (VO) Asan Agibetov, PhD. Deep Learning 861.061 Basic Lecture - Complex Systems & Artificial Intelligence 2017/18 (VO) Asan Agibetov, PhD asan.agibetov@meduniwien.ac.at Medical University of Vienna Center for Medical Statistics,

More information

Deep Learning With Noise

Deep Learning With Noise Deep Learning With Noise Yixin Luo Computer Science Department Carnegie Mellon University yixinluo@cs.cmu.edu Fan Yang Department of Mathematical Sciences Carnegie Mellon University fanyang1@andrew.cmu.edu

More information

Structured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov

Structured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov Structured Light II Johannes Köhler Johannes.koehler@dfki.de Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov Introduction Previous lecture: Structured Light I Active Scanning Camera/emitter

More information

CS 4510/9010 Applied Machine Learning. Neural Nets. Paula Matuszek Fall copyright Paula Matuszek 2016

CS 4510/9010 Applied Machine Learning. Neural Nets. Paula Matuszek Fall copyright Paula Matuszek 2016 CS 4510/9010 Applied Machine Learning 1 Neural Nets Paula Matuszek Fall 2016 Neural Nets, the very short version 2 A neural net consists of layers of nodes, or neurons, each of which has an activation

More information

Knowledge Discovery and Data Mining. Neural Nets. A simple NN as a Mathematical Formula. Notes. Lecture 13 - Neural Nets. Tom Kelsey.

Knowledge Discovery and Data Mining. Neural Nets. A simple NN as a Mathematical Formula. Notes. Lecture 13 - Neural Nets. Tom Kelsey. Knowledge Discovery and Data Mining Lecture 13 - Neural Nets Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-13-NN

More information

Knowledge Discovery and Data Mining

Knowledge Discovery and Data Mining Knowledge Discovery and Data Mining Lecture 13 - Neural Nets Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-13-NN

More information

Texture Image Segmentation using FCM

Texture Image Segmentation using FCM Proceedings of 2012 4th International Conference on Machine Learning and Computing IPCSIT vol. 25 (2012) (2012) IACSIT Press, Singapore Texture Image Segmentation using FCM Kanchan S. Deshmukh + M.G.M

More information

To be Bernoulli or to be Gaussian, for a Restricted Boltzmann Machine

To be Bernoulli or to be Gaussian, for a Restricted Boltzmann Machine 2014 22nd International Conference on Pattern Recognition To be Bernoulli or to be Gaussian, for a Restricted Boltzmann Machine Takayoshi Yamashita, Masayuki Tanaka, Eiji Yoshida, Yuji Yamauchi and Hironobu

More information

CMU Lecture 18: Deep learning and Vision: Convolutional neural networks. Teacher: Gianni A. Di Caro

CMU Lecture 18: Deep learning and Vision: Convolutional neural networks. Teacher: Gianni A. Di Caro CMU 15-781 Lecture 18: Deep learning and Vision: Convolutional neural networks Teacher: Gianni A. Di Caro DEEP, SHALLOW, CONNECTED, SPARSE? Fully connected multi-layer feed-forward perceptrons: More powerful

More information

The exam is closed book, closed notes except your one-page cheat sheet.

The exam is closed book, closed notes except your one-page cheat sheet. CS 189 Fall 2015 Introduction to Machine Learning Final Please do not turn over the page before you are instructed to do so. You have 2 hours and 50 minutes. Please write your initials on the top-right

More information

A Deep Learning primer

A Deep Learning primer A Deep Learning primer Riccardo Zanella r.zanella@cineca.it SuperComputing Applications and Innovation Department 1/21 Table of Contents Deep Learning: a review Representation Learning methods DL Applications

More information

A Novel Image Super-resolution Reconstruction Algorithm based on Modified Sparse Representation

A Novel Image Super-resolution Reconstruction Algorithm based on Modified Sparse Representation , pp.162-167 http://dx.doi.org/10.14257/astl.2016.138.33 A Novel Image Super-resolution Reconstruction Algorithm based on Modified Sparse Representation Liqiang Hu, Chaofeng He Shijiazhuang Tiedao University,

More information

CHAPTER 6 HYBRID AI BASED IMAGE CLASSIFICATION TECHNIQUES

CHAPTER 6 HYBRID AI BASED IMAGE CLASSIFICATION TECHNIQUES CHAPTER 6 HYBRID AI BASED IMAGE CLASSIFICATION TECHNIQUES 6.1 INTRODUCTION The exploration of applications of ANN for image classification has yielded satisfactory results. But, the scope for improving

More information

Consistent Line Clusters for Building Recognition in CBIR

Consistent Line Clusters for Building Recognition in CBIR Consistent Line Clusters for Building Recognition in CBIR Yi Li and Linda G. Shapiro Department of Computer Science and Engineering University of Washington Seattle, WA 98195-250 shapiro,yi @cs.washington.edu

More information

A Fast Learning Algorithm for Deep Belief Nets

A Fast Learning Algorithm for Deep Belief Nets A Fast Learning Algorithm for Deep Belief Nets Geoffrey E. Hinton, Simon Osindero Department of Computer Science University of Toronto, Toronto, Canada Yee-Whye Teh Department of Computer Science National

More information

Optimizing the Deblocking Algorithm for. H.264 Decoder Implementation

Optimizing the Deblocking Algorithm for. H.264 Decoder Implementation Optimizing the Deblocking Algorithm for H.264 Decoder Implementation Ken Kin-Hung Lam Abstract In the emerging H.264 video coding standard, a deblocking/loop filter is required for improving the visual

More information

XES Tensorflow Process Prediction using the Tensorflow Deep-Learning Framework

XES Tensorflow Process Prediction using the Tensorflow Deep-Learning Framework XES Tensorflow Process Prediction using the Tensorflow Deep-Learning Framework Demo Paper Joerg Evermann 1, Jana-Rebecca Rehse 2,3, and Peter Fettke 2,3 1 Memorial University of Newfoundland 2 German Research

More information