Data Compression. The Encoder and PCA
|
|
- Fay Lewis
- 6 years ago
- Views:
Transcription
1 Data Compression The Encoder and PCA Neural network techniques have been shown useful in the area of data compression. In general, data compression can be lossless compression or lossy compression. In the latter, some portion of the information represented is actually lost. JPEG and M PEG (video & audio) compression standards are examples of lossy compression whereas LZW and 'packbits' are lossless. Neural net techniques can be applied to achieve both lossless and lossy compression. The following is a closer look at examples of different neural net based compression techniques. 1
2 The Encoder Self-supervised backpropagation The input is reproduced on the output Hidden layer compresses data Only hidden layer outputs transmitted Output layer used for decoding The encoder is a multi-layer perceptron, trained to act as an autoassociator, using backpropagation. 2
3 The Encoder The net is trained to produce the same output pattern that appears on the input. This is also known as self-supervised backpropagation. The aim is to reproduce the input pattern on the output, but using as few hidden layer neurons as possible. The output of the hidden layer then becomes the data to be transmitted. The "compressed" data is decoded at the receiver using the weights on the output layer. The illustration shows how an n dimensional input pattern can be transmitted using less than n inputs (since there are less than n hidden units). 3
4 The Encoder Lossless compression N orthogonal input patterns can be mapped onto log 2 N hidden units. Lossy compression N < log 2 N It is known (Rumelhart & M cclelland, 1986) that a set of N orthogonal input patterns can be mapped onto log 2 N hidden units. Thus, a figure of log 2 N can be taken as a theoretical minimum number of hidden units to achieve lossless compression. 4
5 The Encoder Cottrell et. Al. (1987) Image compression Greyscale 8 bit image, any dimensions Network size : 64 in, 64 out and 16 hidden Image processed in 8x8 patches. An example of this approach for image compression was investigated by Cottrell et al. (1987). The aim here was to compress an image (of any size). Their approach used a network with 64 inputs (representing an 8x8 image patch), 16 hidden units and 64 outputs. Each input represented a 256 level. 5
6 The Encoder Near state of the art results obtained! 64 greyscale pixels compressed by 16 hidden units. 150,000 training patterns Compression is image dependent however. Encode & transmit first 8x8 patch The net was trained using 150,000 presentations of input taken randomly from 8x8 patches of the image. Applying the net to each 8x8 non-overlapping patch of the image Cottrell obtained near state of the art compression results! Note however that compression was very much tuned to the actual image compressed and that results with other kinds of images were less impressive. 6
7 Principal Component Analysis PCA is dimensionality reduction m bit data converted to n bit data where n < m Another way to view data compression is to regard it as a reduction in dimensionality. That is, can a representation of a set of patterns expressed using n bits of information, be adequately described using m bits, where m is less than n? The goal is to effectively represent the same data using a reduced set of features. Given a set of data then principal component analysis, as we have already seen, attempts to identify axes (or principal components) along which the data varies the greatest. 7
8 PCA By definition PCA is lossy compression Reduction in number of features used to represent data Which features to keep and which to remove? 8
9 PCA Principal components Are axes along which data varies the most 1 st principal component exhibits greatest variance 2 nd principal component exhibits next greatest variance Etc. The first principal component is regarded to be the axis, which exhibits the greatest variance in data. The second component is orthogonal to the first and shows the second greatest variance of data, the third is orthogonal to the first two and so on. 9
10 PCA 2 nd component orthogonal to 1 st 3 rd orthogonal to 2 nd Etc. Original axes - clusters difficult to discriminate Original Data 2 nd PC 1 st PC Principal components - easier to discriminate 10
11 PCA The Hebb Rule Oja 1992 : a single neuron, can be trained to find the 1 st PC Sanger 1989 : in general m neurons can be trained to find the first m PCs The generalized Hebbian algorithm (GHA) In terms of neural nets, a Hebb like learning rule can be used to train a single neuron so that the weights converge to the principal component of a distribution (Oja, 1992). In general, a layer of m neurons, can be trained using a "generalized Hebbian algorithm" (GHA) to find the first m principal components of a set of input data (Sanger 1989). 11
12 PCA & Image Compression Haykin 1994 Describes the GHA for image compression Example Input image 256 x 256, each pixel with 256 grey levels PCA network 8 neurons, each with an 8 x 8 receptive field Haykin describes an application of GHA for image compression. A 256 by 256 image, where each pixel had 256 grey levels was chosen for encoding. The image was coded using a linear feedforward network of 8 neurons, each with 64 inputs. Training was performed by presenting the net with data taken from 8x8 non-overlapping patches of the image. To allow convergence, the image was scanned from left to right, and top to bottom, twice. The 8 neurons represent the first 8 eigenvectors obtained by convergence. (Sanger's rule). 12
13 PCA & Image Compression Haykin 1994 Processing Image scanned top-to-bottom, and left-to-right. The neurons let to converge. The 8 neurons represents the first 8 eigenvectors. 13
14 PCA & Image Compression Example input image From Haykin, S., "Neural Networks - A Comprehensive Foundation", 1994 Once the weights of the network had converged, they were used to encode the image (shown above) for transmission. 14
15 PCA & Image Compression Encoding details Each 8 x 8 block multiplied by each neuron This gives 8 coefficients Coefficients transmitted. In Haykin s example, 23 bits needed. I.e. 23 bits encoded an 8x8x8 image patch. Transmission Each 8x8 block of the image was multiplied by the weights from each of the eight neurons (I,e. applied to each neuron). This generated 8 outputs or coefficients. The coefficient from each neuron was transmitted. The number of bits chosen to represent each coefficient is determined by variance of the coefficient over the whole image. (I.e. a larger number of bits are needed to represent something that varies a lot, rather than something that varies a little). In the example described in Haykin, this required 23 bits to code the outputs of the 8 neurons. (That is, 23 bits were required to encode each 8x8 block of pixels, where each pixel was represented using 8bits). 15
16 PCA & Image Compression The weights of the 8 neurons From Haykin, S., "Neural Networks - A Comprehensive Foundation", 1994 The illustration above shows the weights obtained by each of the eight neurons. In the diagram, light areas depict positive weights and dark areas negative (or inhibitory) weights. 16
17 PCA & Image Compression Decoding details Neurons used to decode transmitted coefficients. Weights x coefficient = 8 x 8 patch reconstructed Receiving (decoding) The image was reconstructed from the transmitted coefficients using the neurons again. This time however, the weights of each neuron were multiplied by their coefficient and then added together to reconstruct each 8x8 patch of the image. The following illustration shows the weights obtained by each of the eight neurons. In the diagram, light areas depict positive weights and dark areas negative (or inhibitory) weights. 17
18 PCA & Image Compression! Transmission 8x8 patch of image 8 8 Weights of each neuron represent one of first eight principal components of image data. Obtained using Sanger's rule. 18
19 PCA & Image Compression Example output image Input Image Output Image From Haykin, S., "Neural Networks - A Comprehensive Foundation",
Neural Networks for unsupervised learning From Principal Components Analysis to Autoencoders to semantic hashing
Neural Networks for unsupervised learning From Principal Components Analysis to Autoencoders to semantic hashing feature 3 PC 3 Beate Sick Many slides are taken form Hinton s great lecture on NN: https://www.coursera.org/course/neuralnets
More informationNeural Networks for Machine Learning. Lecture 15a From Principal Components Analysis to Autoencoders
Neural Networks for Machine Learning Lecture 15a From Principal Components Analysis to Autoencoders Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed Principal Components
More informationImage coding and compression
Chapter 2 Image coding and compression 2. Lossless and lossy compression We have seen that image files can be very large. It is thus important for reasons both of storage and file transfer to make these
More informationWhy Normalizing v? Why did we normalize v on the right side? Because we want the length of the left side to be the eigenvalue
Why Normalizing v? Why did we normalize v on the right side? Because we want the length of the left side to be the eigenvalue 1 CCI PCA Algorithm (1) 2 CCI PCA Algorithm (2) 3 PCA from the FERET Face Image
More informationCenter for Automation and Autonomous Complex Systems. Computer Science Department, Tulane University. New Orleans, LA June 5, 1991.
Two-phase Backpropagation George M. Georgiou Cris Koutsougeras Center for Automation and Autonomous Complex Systems Computer Science Department, Tulane University New Orleans, LA 70118 June 5, 1991 Abstract
More informationSlides adapted from Marshall Tappen and Bryan Russell. Algorithms in Nature. Non-negative matrix factorization
Slides adapted from Marshall Tappen and Bryan Russell Algorithms in Nature Non-negative matrix factorization Dimensionality Reduction The curse of dimensionality: Too many features makes it difficult to
More informationClassifier C-Net. 2D Projected Images of 3D Objects. 2D Projected Images of 3D Objects. Model I. Model II
Advances in Neural Information Processing Systems 7. (99) The MIT Press, Cambridge, MA. pp.949-96 Unsupervised Classication of 3D Objects from D Views Satoshi Suzuki Hiroshi Ando ATR Human Information
More informationNovel Lossy Compression Algorithms with Stacked Autoencoders
Novel Lossy Compression Algorithms with Stacked Autoencoders Anand Atreya and Daniel O Shea {aatreya, djoshea}@stanford.edu 11 December 2009 1. Introduction 1.1. Lossy compression Lossy compression is
More informationCSC321: Neural Networks. Lecture 13: Learning without a teacher: Autoencoders and Principal Components Analysis. Geoffrey Hinton
CSC321: Neural Networks Lecture 13: Learning without a teacher: Autoencoders and Principal Components Analysis Geoffrey Hinton Three problems with backpropagation Where does the supervision come from?
More informationSlide07 Haykin Chapter 9: Self-Organizing Maps
Slide07 Haykin Chapter 9: Self-Organizing Maps CPSC 636-600 Instructor: Yoonsuck Choe Spring 2012 Introduction Self-organizing maps (SOM) is based on competitive learning, where output neurons compete
More informationCSC 411: Lecture 14: Principal Components Analysis & Autoencoders
CSC 411: Lecture 14: Principal Components Analysis & Autoencoders Raquel Urtasun & Rich Zemel University of Toronto Nov 4, 2015 Urtasun & Zemel (UofT) CSC 411: 14-PCA & Autoencoders Nov 4, 2015 1 / 18
More informationNeural Network based textural labeling of images in multimedia applications
Neural Network based textural labeling of images in multimedia applications S.A. Karkanis +, G.D. Magoulas +, and D.A. Karras ++ + University of Athens, Dept. of Informatics, Typa Build., Panepistimiopolis,
More informationCOMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS
COMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS Toomas Kirt Supervisor: Leo Võhandu Tallinn Technical University Toomas.Kirt@mail.ee Abstract: Key words: For the visualisation
More informationMachine Learning for Physicists Lecture 6. Summer 2017 University of Erlangen-Nuremberg Florian Marquardt
Machine Learning for Physicists Lecture 6 Summer 2017 University of Erlangen-Nuremberg Florian Marquardt Channels MxM image MxM image K K 3 channels conv 6 channels in any output channel, each pixel receives
More informationUnsupervised Learning
Unsupervised Learning Learning without a teacher No targets for the outputs Networks which discover patterns, correlations, etc. in the input data This is a self organisation Self organising networks An
More informationCOMP 551 Applied Machine Learning Lecture 16: Deep Learning
COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all
More informationCSC 411: Lecture 14: Principal Components Analysis & Autoencoders
CSC 411: Lecture 14: Principal Components Analysis & Autoencoders Richard Zemel, Raquel Urtasun and Sanja Fidler University of Toronto Zemel, Urtasun, Fidler (UofT) CSC 411: 14-PCA & Autoencoders 1 / 18
More informationNeuro-Fuzzy Computing
CSE53 Neuro-Fuzzy Computing Tutorial/Assignment 3: Unsupervised Learning About this tutorial The objective of this tutorial is to study unsupervised learning, in particular: (Generalized) Hebbian learning.
More informationLab 8 CSC 5930/ Computer Vision
Lab 8 CSC 5930/9010 - Computer Vision Description: One way to effectively train a neural network with multiple layers is by training one layer at a time. You can achieve this by training a special type
More informationEfficient Visual Coding: From Retina To V2
Efficient Visual Coding: From Retina To V Honghao Shan Garrison Cottrell Computer Science and Engineering UCSD La Jolla, CA 9093-0404 shanhonghao@gmail.com, gary@ucsd.edu Abstract The human visual system
More informationDimension Reduction CS534
Dimension Reduction CS534 Why dimension reduction? High dimensionality large number of features E.g., documents represented by thousands of words, millions of bigrams Images represented by thousands of
More informationYuki Osada Andrew Cannon
Yuki Osada Andrew Cannon 1 Humans are an intelligent species One feature is the ability to learn The ability to learn comes down to the brain The brain learns from experience Research shows that the brain
More informationDeep Learning. Volker Tresp Summer 2014
Deep Learning Volker Tresp Summer 2014 1 Neural Network Winter and Revival While Machine Learning was flourishing, there was a Neural Network winter (late 1990 s until late 2000 s) Around 2010 there
More informationContents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation
Contents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation Learning 4 Supervised Learning 4 Unsupervised Learning 4
More informationGray-Level Reduction Using Local Spatial Features
Computer Vision and Image Understanding 78, 336 350 (2000) doi:10.1006/cviu.2000.0838, available online at http://www.idealibrary.com on Gray-Level Reduction Using Local Spatial Features Nikos Papamarkos
More informationData Mining. Neural Networks
Data Mining Neural Networks Goals for this Unit Basic understanding of Neural Networks and how they work Ability to use Neural Networks to solve real problems Understand when neural networks may be most
More informationCharacter Recognition Using Convolutional Neural Networks
Character Recognition Using Convolutional Neural Networks David Bouchain Seminar Statistical Learning Theory University of Ulm, Germany Institute for Neural Information Processing Winter 2006/2007 Abstract
More informationNeural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer
More informationClustering and Dimensionality Reduction
Clustering and Dimensionality Reduction Some material on these is slides borrowed from Andrew Moore's excellent machine learning tutorials located at: Data Mining Automatically extracting meaning from
More informationCOMPUTATIONAL INTELLIGENCE
COMPUTATIONAL INTELLIGENCE Radial Basis Function Networks Adrian Horzyk Preface Radial Basis Function Networks (RBFN) are a kind of artificial neural networks that use radial basis functions (RBF) as activation
More informationDr. Qadri Hamarsheh Supervised Learning in Neural Networks (Part 1) learning algorithm Δwkj wkj Theoretically practically
Supervised Learning in Neural Networks (Part 1) A prescribed set of well-defined rules for the solution of a learning problem is called a learning algorithm. Variety of learning algorithms are existing,
More informationCIE L*a*b* color model
CIE L*a*b* color model To further strengthen the correlation between the color model and human perception, we apply the following non-linear transformation: with where (X n,y n,z n ) are the tristimulus
More informationRichard S. Zemel 1 Georey E. Hinton North Torrey Pines Rd. Toronto, ONT M5S 1A4. Abstract
Developing Population Codes By Minimizing Description Length Richard S Zemel 1 Georey E Hinton University oftoronto & Computer Science Department The Salk Institute, CNL University oftoronto 0 North Torrey
More informationCS 4510/9010 Applied Machine Learning. Deep Learning. Paula Matuszek Fall copyright Paula Matuszek 2016
CS 4510/9010 Applied Machine Learning 1 Deep Learning Paula Matuszek Fall 2016 Beyond Simple Neural Nets 2 In the last few ideas we have seen some surprisingly rapid progress in some areas of AI Image
More informationCIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]
CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, 2015. 11:59pm, PDF to Canvas [100 points] Instructions. Please write up your responses to the following problems clearly and concisely.
More informationImage Compression Using SOFM
Image Compression Using SOFM Ankit Aggarwal (03d05009) Anshu Agrawal (03005006) November 12, 2006 Why Image Compression? Application of data compression on digital images. Computer images are extremely
More informationCPSC 340: Machine Learning and Data Mining. Deep Learning Fall 2018
CPSC 340: Machine Learning and Data Mining Deep Learning Fall 2018 Last Time: Multi-Dimensional Scaling Multi-dimensional scaling (MDS): Non-parametric visualization: directly optimize the z i locations.
More informationGrundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Unsupervised learning Daniel Hennes 29.01.2018 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Supervised learning Regression (linear
More informationHW Assignment 3 (Due by 9:00am on Mar 6)
HW Assignment 3 (Due by 9:00am on Mar 6) 1 Theory (150 points) 1. [Tied Weights, 50 points] Write down the gradient computation for a (non-linear) auto-encoder with tied weights i.e., W (2) = (W (1) )
More informationCS325 Artificial Intelligence Ch. 20 Unsupervised Machine Learning
CS325 Artificial Intelligence Cengiz Spring 2013 Unsupervised Learning Missing teacher No labels, y Just input data, x What can you learn with it? Unsupervised Learning Missing teacher No labels, y Just
More informationImageNet Classification with Deep Convolutional Neural Networks
ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky Ilya Sutskever Geoffrey Hinton University of Toronto Canada Paper with same name to appear in NIPS 2012 Main idea Architecture
More informationA Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images Marc Aurelio Ranzato Yann LeCun Courant Institute of Mathematical Sciences New York University - New York, NY 10003 Abstract
More informationIMPROVEMENTS TO THE BACKPROPAGATION ALGORITHM
Annals of the University of Petroşani, Economics, 12(4), 2012, 185-192 185 IMPROVEMENTS TO THE BACKPROPAGATION ALGORITHM MIRCEA PETRINI * ABSTACT: This paper presents some simple techniques to improve
More informationLossless Image Compression with Lossy Image Using Adaptive Prediction and Arithmetic Coding
Lossless Image Compression with Lossy Image Using Adaptive Prediction and Arithmetic Coding Seishi Taka" and Mikio Takagi Institute of Industrial Science, University of Tokyo Abstract Lossless gray scale
More informationMultimedia Data and Its Encoding
Lecture 13 Multimedia Data and Its Encoding M. Adnan Quaium Assistant Professor Department of Electrical and Electronic Engineering Ahsanullah University of Science and Technology Room 4A07 Email adnan.eee@aust.edu
More informationRadial Basis Function Neural Network Classifier
Recognition of Unconstrained Handwritten Numerals by a Radial Basis Function Neural Network Classifier Hwang, Young-Sup and Bang, Sung-Yang Department of Computer Science & Engineering Pohang University
More informationCompression Part 2 Lossy Image Compression (JPEG) Norm Zeck
Compression Part 2 Lossy Image Compression (JPEG) General Compression Design Elements 2 Application Application Model Encoder Model Decoder Compression Decompression Models observe that the sensors (image
More informationSo, what is data compression, and why do we need it?
In the last decade we have been witnessing a revolution in the way we communicate 2 The major contributors in this revolution are: Internet; The explosive development of mobile communications; and The
More informationA Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images Marc Aurelio Ranzato Yann LeCun Courant Institute of Mathematical Sciences New York University - New York, NY 10003 Abstract
More informationMultithresholding of color and gray-level images through a neural network technique
Image and Vision Computing 8 (2000) 23 222 www.elsevier.com/locate/imavis Multithresholding of color and gray-level images through a neural network technique N. Papamarkos*, C. Strouthopoulos, I. Andreadis
More informationClustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
More informationSupport Vector Machines
Support Vector Machines RBF-networks Support Vector Machines Good Decision Boundary Optimization Problem Soft margin Hyperplane Non-linear Decision Boundary Kernel-Trick Approximation Accurancy Overtraining
More informationCS 6501: Deep Learning for Computer Graphics. Training Neural Networks II. Connelly Barnes
CS 6501: Deep Learning for Computer Graphics Training Neural Networks II Connelly Barnes Overview Preprocessing Initialization Vanishing/exploding gradients problem Batch normalization Dropout Additional
More information3 Data Storage 3.1. Foundations of Computer Science Cengage Learning
3 Data Storage 3.1 Foundations of Computer Science Cengage Learning Objectives After studying this chapter, the student should be able to: List five different data types used in a computer. Describe how
More informationVisual object classification by sparse convolutional neural networks
Visual object classification by sparse convolutional neural networks Alexander Gepperth 1 1- Ruhr-Universität Bochum - Institute for Neural Dynamics Universitätsstraße 150, 44801 Bochum - Germany Abstract.
More informationA Fast Personal Palm print Authentication based on 3D-Multi Wavelet Transformation
A Fast Personal Palm print Authentication based on 3D-Multi Wavelet Transformation * A. H. M. Al-Helali, * W. A. Mahmmoud, and * H. A. Ali * Al- Isra Private University Email: adnan_hadi@yahoo.com Abstract:
More informationHaar Wavelet Image Compression
Math 57 Haar Wavelet Image Compression. Preliminaries Haar wavelet compression is an efficient way to perform both lossless and lossy image compression. It relies on averaging and differencing the values
More informationIMAGE COMPRESSION. October 7, ICSY Lab, University of Kaiserslautern, Germany
Lossless Compression Multimedia File Formats Lossy Compression IMAGE COMPRESSION 69 Basic Encoding Steps 70 JPEG (Overview) Image preparation and coding (baseline system) 71 JPEG (Enoding) 1) select color
More informationComputational Photography Denoising
Computational Photography Denoising Jongmin Baek CS 478 Lecture Feb 13, 2012 Announcements Term project proposal Due Wednesday Proposal presentation Next Wednesday Send us your slides (Keynote, PowerPoint,
More informationNon-linear Point Distribution Modelling using a Multi-layer Perceptron
Non-linear Point Distribution Modelling using a Multi-layer Perceptron P.D. Sozou, T.F. Cootes, C.J. Taylor and E.C. Di Mauro Department of Medical Biophysics University of Manchester M13 9PT email: pds@svl.smb.man.ac.uk
More informationProgramming Exercise 7: K-means Clustering and Principal Component Analysis
Programming Exercise 7: K-means Clustering and Principal Component Analysis Machine Learning May 13, 2012 Introduction In this exercise, you will implement the K-means clustering algorithm and apply it
More informationThe principal components of natural images. Peter J.B. Hancock, Roland J. Baddeley and Leslie S. Smith. University of Stirling, Scotland FK9 4LA
The principal components of natural images Peter J.B. Hancock, Roland J. Baddeley and Leslie S. Smith Centre for Cognitive and Computational Neuroscience Departments of Psychology and Computing Science
More information4.12 Generalization. In back-propagation learning, as many training examples as possible are typically used.
1 4.12 Generalization In back-propagation learning, as many training examples as possible are typically used. It is hoped that the network so designed generalizes well. A network generalizes well when
More informationArtificial neural networks are the paradigm of connectionist systems (connectionism vs. symbolism)
Artificial Neural Networks Analogy to biological neural systems, the most robust learning systems we know. Attempt to: Understand natural biological systems through computational modeling. Model intelligent
More informationAPPLICATION OF RECIRCULATION NEURAL NETWORK AND PRINCIPAL COMPONENT ANALYSIS FOR FACE RECOGNITION
APPLICATION OF RECIRCULATION NEURAL NETWORK AND PRINCIPAL COMPONENT ANALYSIS FOR FACE RECOGNITION Dmitr Brliuk and Valer Starovoitov Institute of Engineering Cbernetics, Laborator of Image Processing and
More informationPrincipal Component Analysis (PCA) is a most practicable. statistical technique. Its application plays a major role in many
CHAPTER 3 PRINCIPAL COMPONENT ANALYSIS ON EIGENFACES 2D AND 3D MODEL 3.1 INTRODUCTION Principal Component Analysis (PCA) is a most practicable statistical technique. Its application plays a major role
More informationRecognition: Face Recognition. Linda Shapiro EE/CSE 576
Recognition: Face Recognition Linda Shapiro EE/CSE 576 1 Face recognition: once you ve detected and cropped a face, try to recognize it Detection Recognition Sally 2 Face recognition: overview Typical
More informationData Mining. Kohonen Networks. Data Mining Course: Sharif University of Technology 1
Data Mining Kohonen Networks Data Mining Course: Sharif University of Technology 1 Self-Organizing Maps Kohonen Networks developed in 198 by Tuevo Kohonen Initially applied to image and sound analysis
More informationMachine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves
Machine Learning A 708.064 11W 1sst KU Exercises Problems marked with * are optional. 1 Conditional Independence I [2 P] a) [1 P] Give an example for a probability distribution P (A, B, C) that disproves
More informationImage Compression: An Artificial Neural Network Approach
Image Compression: An Artificial Neural Network Approach Anjana B 1, Mrs Shreeja R 2 1 Department of Computer Science and Engineering, Calicut University, Kuttippuram 2 Department of Computer Science and
More informationFunction approximation using RBF network. 10 basis functions and 25 data points.
1 Function approximation using RBF network F (x j ) = m 1 w i ϕ( x j t i ) i=1 j = 1... N, m 1 = 10, N = 25 10 basis functions and 25 data points. Basis function centers are plotted with circles and data
More informationIntroduction to Machine Learning Prof. Anirban Santara Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur
Introduction to Machine Learning Prof. Anirban Santara Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture 14 Python Exercise on knn and PCA Hello everyone,
More informationCombined Weak Classifiers
Combined Weak Classifiers Chuanyi Ji and Sheng Ma Department of Electrical, Computer and System Engineering Rensselaer Polytechnic Institute, Troy, NY 12180 chuanyi@ecse.rpi.edu, shengm@ecse.rpi.edu Abstract
More informationSupport Vector Machines
Support Vector Machines RBF-networks Support Vector Machines Good Decision Boundary Optimization Problem Soft margin Hyperplane Non-linear Decision Boundary Kernel-Trick Approximation Accurancy Overtraining
More informationCS228: Project Report Boosted Decision Stumps for Object Recognition
CS228: Project Report Boosted Decision Stumps for Object Recognition Mark Woodward May 5, 2011 1 Introduction This project is in support of my primary research focus on human-robot interaction. In order
More informationImage Compression Technique
Volume 2 Issue 2 June 2014 ISSN: 2320-9984 (Online) International Journal of Modern Engineering & Management Research Shabbir Ahmad Department of Computer Science and Engineering Bhagwant University, Ajmer
More informationImage Coding and Data Compression
Image Coding and Data Compression Biomedical Images are of high spatial resolution and fine gray-scale quantisiation Digital mammograms: 4,096x4,096 pixels with 12bit/pixel 32MB per image Volume data (CT
More informationResearch Article A Novel Image Compression Method Based on Classified Energy and Pattern Building Blocks
Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing Volume 2011, Article ID 730694, 20 pages doi:10.1155/2011/730694 Research Article A Novel Image Compression Method Based
More informationIndex. 1. Motivation 2. Background 3. JPEG Compression The Discrete Cosine Transformation Quantization Coding 4. MPEG 5.
Index 1. Motivation 2. Background 3. JPEG Compression The Discrete Cosine Transformation Quantization Coding 4. MPEG 5. Literature Lossy Compression Motivation To meet a given target bit-rate for storage
More informationISSN (ONLINE): , VOLUME-3, ISSUE-1,
PERFORMANCE ANALYSIS OF LOSSLESS COMPRESSION TECHNIQUES TO INVESTIGATE THE OPTIMUM IMAGE COMPRESSION TECHNIQUE Dr. S. Swapna Rani Associate Professor, ECE Department M.V.S.R Engineering College, Nadergul,
More information2. Neural network basics
2. Neural network basics Next commonalities among different neural networks are discussed in order to get started and show which structural parts or concepts appear in almost all networks. It is presented
More informationUsing Machine Learning for Classification of Cancer Cells
Using Machine Learning for Classification of Cancer Cells Camille Biscarrat University of California, Berkeley I Introduction Cell screening is a commonly used technique in the development of new drugs.
More informationD.A. Karras 1, S.A. Karkanis 2, D. Iakovidis 2, D. E. Maroulis 2 and B.G. Mertzios , Greece,
NIMIA-SC2001-2001 NATO Advanced Study Institute on Neural Networks for Instrumentation, Measurement, and Related Industrial Applications: Study Cases Crema, Italy, 9-20 October 2001 Support Vector Machines
More informationThe Fly & Anti-Fly Missile
The Fly & Anti-Fly Missile Rick Tilley Florida State University (USA) rt05c@my.fsu.edu Abstract Linear Regression with Gradient Descent are used in many machine learning applications. The algorithms are
More informationNeural Network Neurons
Neural Networks Neural Network Neurons 1 Receives n inputs (plus a bias term) Multiplies each input by its weight Applies activation function to the sum of results Outputs result Activation Functions Given
More informationIMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS
IMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS BOGDAN M.WILAMOWSKI University of Wyoming RICHARD C. JAEGER Auburn University ABSTRACT: It is shown that by introducing special
More informationWhy MultiLayer Perceptron/Neural Network? Objective: Attributes:
Why MultiLayer Perceptron/Neural Network? Neural networks, with their remarkable ability to derive meaning from complicated or imprecise data, can be used to extract patterns and detect trends that are
More informationDeep Generative Models Variational Autoencoders
Deep Generative Models Variational Autoencoders Sudeshna Sarkar 5 April 2017 Generative Nets Generative models that represent probability distributions over multiple variables in some way. Directed Generative
More informationAn introduction to JPEG compression using MATLAB
An introduction to JPEG compression using MATLAB Arno Swart 30 October, 2003 1 Introduction This document describes the popular JPEG still image coding format. The aim is to compress images while maintaining
More informationDimension reduction : PCA and Clustering
Dimension reduction : PCA and Clustering By Hanne Jarmer Slides by Christopher Workman Center for Biological Sequence Analysis DTU The DNA Array Analysis Pipeline Array design Probe design Question Experimental
More informationNeural Networks. Single-layer neural network. CSE 446: Machine Learning Emily Fox University of Washington March 10, /10/2017
3/0/207 Neural Networks Emily Fox University of Washington March 0, 207 Slides adapted from Ali Farhadi (via Carlos Guestrin and Luke Zettlemoyer) Single-layer neural network 3/0/207 Perceptron as a neural
More informationMachine Learning : Clustering, Self-Organizing Maps
Machine Learning Clustering, Self-Organizing Maps 12/12/2013 Machine Learning : Clustering, Self-Organizing Maps Clustering The task: partition a set of objects into meaningful subsets (clusters). The
More informationFeature Extraction and Learning for RSSI based Indoor Device Localization
Feature Extraction and Learning for RSSI based Indoor Device Localization Stavros Timotheatos, Grigorios Tsagkatakis, Panagiotis Tsakalides and Panos Trahanias Institute of Computer Science Foundation
More informationArtificial Neural Networks (Feedforward Nets)
Artificial Neural Networks (Feedforward Nets) y w 03-1 w 13 y 1 w 23 y 2 w 01 w 21 w 22 w 02-1 w 11 w 12-1 x 1 x 2 6.034 - Spring 1 Single Perceptron Unit y w 0 w 1 w n w 2 w 3 x 0 =1 x 1 x 2 x 3... x
More informationMachine Learning Classifiers and Boosting
Machine Learning Classifiers and Boosting Reading Ch 18.6-18.12, 20.1-20.3.2 Outline Different types of learning problems Different types of learning algorithms Supervised learning Decision trees Naïve
More informationArtificial Neuron Modelling Based on Wave Shape
Artificial Neuron Modelling Based on Wave Shape Kieran Greer, Distributed Computing Systems, Belfast, UK. http://distributedcomputingsystems.co.uk Version 1.2 Abstract This paper describes a new model
More informationNotes on Multilayer, Feedforward Neural Networks
Notes on Multilayer, Feedforward Neural Networks CS425/528: Machine Learning Fall 2012 Prepared by: Lynne E. Parker [Material in these notes was gleaned from various sources, including E. Alpaydin s book
More informationCHAPTER 6. 6 Huffman Coding Based Image Compression Using Complex Wavelet Transform. 6.3 Wavelet Transform based compression technique 106
CHAPTER 6 6 Huffman Coding Based Image Compression Using Complex Wavelet Transform Page No 6.1 Introduction 103 6.2 Compression Techniques 104 103 6.2.1 Lossless compression 105 6.2.2 Lossy compression
More informationJPEG. Table of Contents. Page 1 of 4
Page 1 of 4 JPEG JPEG is an acronym for "Joint Photographic Experts Group". The JPEG standard is an international standard for colour image compression. JPEG is particularly important for multimedia applications
More information11/14/2010 Intelligent Systems and Soft Computing 1
Lecture 8 Artificial neural networks: Unsupervised learning Introduction Hebbian learning Generalised Hebbian learning algorithm Competitive learning Self-organising computational map: Kohonen network
More information