DEEP LEARNING FOR BRAIN DECODING
|
|
- Oswald Casey
- 6 years ago
- Views:
Transcription
1 DEEP LEARNING FOR BRAIN DECODING Orhan Firat Ilke Oztekin Fatos T. Yarman Vural Department of Computer Engineering, Middle East Technical University, Ankara, Turkey Department of Psychology, Koc University, Istanbul, Turkey ABSTRACT Learning low dimensional embedding spaces (manifolds) for efficient feature representation is crucial for complex and high dimensional input spaces. Functional magnetic resonance imaging (fmri) produces high dimensional input data and with a less then ideal number of labeled samples for a classification task. In this study, we explore deep learning methods for fmri classification tasks in order to reduce dimensions of feature space, along with improving classification performance for brain decoding. We employ sparse autoencoders for unsupervised feature learning, leveraging unlabeled fmri data to learn efficient, non-linear representations as the building blocks of a deep learning architecture by stacking them. Proposed method is tested on a memory encoding/retrieval experiment with ten classes. The results support the efficiency compared to the baseline multi-voxel pattern analysis techniques. Index Terms Deep Learning, Stacked Autoencoders, fmri, MVPA, brain state decoding 1. INTRODUCTION Multi-voxel Pattern Analysis (MVPA) methods have become increasingly popular in the analysis of functional Magnetic Resonance Imaging (fmri) data [1 4]. These methods are employed in many problems, such as hypothesis validation [5], diagnosing disorders [4, 6] and recently brain state decoding, also known as mind reading [1, 7]. The major difficulty of using the fmri data for pattern analysis is the deficiency of the labeled samples in comparison to the high dimensional feature representations. The classical pattern recognition techniques are prone to several problems, such as over-fitting, insignificance and curse-of-dimensionality. In order to overcome the limitations of the classical MVPA methods, the fmri data is restricted a priori to a subset of voxels which obviously results in losing several relational information-channels (relations between voxels) [5 8]. The trade-off between the restriction on number of voxels (region of interest selection) and the need to employ voxels from Thanks to TUBITAK agency for funding with grant no:112e315. whole brain (even redundant and un-informative ones) may result in misleading analyses about the nature of the brain states. The shortcomings of the available techniques necessitates us to study approaches that are robust to the sparsity and information integrity for feature representations. It is well-established that deep learning architectures resemble some similarities with the human brain in terms of scalability. Recent studies on unsupervised feature learning and deep learning propose an alternative to the classical MVPA techniques by making use of the unlabeled data and learning a feature hierarchy automatically [9 11]. Unsupervised feature learning leverages unlabeled data to learn basic patterns (regularities) which can be used in forthcoming supervised tasks [12, 13]. On the other hand, deep learning constructs several layers of features where higher levels capture more abstract forms of variations in data [14]. By combining the deep learning architecture with the unsupervised feature learning backbone, it is possible to discover a low-dimensional feature space. This manifold space can then be used to map high dimensional input data [15, 16]. Deep learning is a rapidly growing area of AI and recently emerging for computational medical imaging [17 20]. A stacked autoencoder (SAE) is used in [19], for organ detection using 4D data by making use of spatial and temporal filters which are learned automatically. A similar SAE is used in [18] to learn a joint feature representation using for MRI, PET and CSF for Alzheimer s disease detection. In [20], a low-dimensional manifold is learned using a stacked convolutional restricted Boltzmann machine for Alzheiher s detection using structural MRI. Image segmentation for MRI is subjected in [17] by a stacked convolutional independent subspace analysis network. In this study, we propose to model the 4-dimensional spatio-temporal fmri data by the features extracted from the unsupervised feature learning method. Then, we explore the deep learning openings for better discrimination of cognitive processes. We conduct a memory encoding and retrieval experiment with ten semantic categories. Then, we use the unlabeled fmri data for unsupervised learning representations along with a stacked sparse-autoencoder to reduce
2 dimensionality for brain state classification. 2. AUTOENCODERS FOR UNSUPERVISED FEATURE LEARNING An autoencoder is a type of neural network which reconstructs its input by setting the target values to be equal to the inputs, x x. An autoencoder consists of two consecutive functions. The first one is an encoder function f θ1 (x) applied on input data x with parameters = {W (1), b (1) } (transition and bias respectively). This function maps the input data to a hidden representation h. The second function is a decoder function g θ2 (f θ1 (x)) which maps the hidden representations to a reconstruction x of input, parametrized by θ 2 = {W (2), b (2) }. Note that, encoding and decoding functions are mappings, such that, f : R N R K and g : R K R N, where N is the input dimensionality and K is the number of neurons in the hidden layer. By enforcing some constraints, (a sparsity [21] or contracting term [22] regarding to hidden layer activations or distorting input respectively [23]) the autoencoder learns a compact and non-linear representation of the input. Therefore, it keeps away from learning an identity function. In this study, we focus on sparse autoencoders having single hidden layers in the encoding/decoding functions by enforcing a sparsity (activation around zero) to hidden unit neurons via the cost function. After learning model parameters Θ = {, θ 2 }, the sparse autoencoder learns the non-linear feature mapping function f which can be used in further classification tasks or feeding the input layer of another autoencoder. A sparse autoencoder having K hidden neurons, is trained in order to minimize squared reconstruction error using backpropagation by minimizing the following cost function, J sparse (Θ) = J NN (Θ) + βjˆρ (1) where J NN (Θ) being regularized neural network cost and Jˆρ regarding to sparsity term. Hyper-parameter β controls the importance of sparsity in the model. In the general, L2 regularized neural network (having 1 hidden layer) cost with m examples is as follows, J NN (Θ) = 1 2m m x (i) x (i) 2 λ + 2 i l,u,v ( W (l) uv ) 2, (2) where first term corresponds to the squared reconstruction error and the second term corresponds to the weight decay term that penalizes large values of transition parameters for all entries u, v. The activation of the each layer l is computed as a (l) = σ(w (l 1) a (l 1) + b (l 1) ) where λ is the regularization parameter, σ is the sigmoid function and. indicates L2 norm and a (0) = x. Note that, f θ1 (x) = a (1) in our model. Further, let ˆρ j = 1 m i a (2) j (x (i) ), (3) be the average activation of hidden unit j over the dataset and enforce the constraint ˆρ j = ρ where ρ is the sparsity parameter (chosen to be close to zero). In order to measure the sparsity cost, Kullback-Leibler divergence between average activation of a unit ˆρ and sparsity parameters ρ is employed. Finally, sparsity term of the overall cost function in (1) is given as follows: Jˆρ = K K KL(ρ ˆρ j ) = ρ log ρˆρ + (1 ρ) log 1 ρ j j j 1 ˆρ j (4) Optimization of the model parameters can be achieved by gradient descent. Since the objective gradient can be computed exactly, it is plausible to use advanced optimization methods. For this reason, we employ a quasi-newton L-BFGS method. Note that, for the inputs that are not in [0 1] range, output layer activations can be set directly as the weighted sum of its pre-activations without passing through a point-wise sigmoid function, meaning a (2) = g θ2 (f θ1 (x)) = W (1) a (1) + b (1). 3. STACKED AUTOENCODERS FOR DEEP LEARNING REPRESENTATIONS Given the unlabeled fmri data, the autoencoder learns nonlinear intermediate representations (codes) to reconstruct the data. In order to achieve a better reconstruction, representations must specifically capture the necessary variations to distinguish examples in the input data. This constitutes the directions of variations on the manifold where the probability mass (the data generating distribution) concentrates [14]. It has also been shown that highly complex functions can be represented with fewer parameters through the composition of many non-linearities, which is a deep architecture as opposed to the shallow ones [9, 15]. Hence by stacking autoencoders (or any non-linear feature mappings), more abstract and complex representations are achieved which also makes the data generating distribution more uniform manifolds. The rationale of unfolding the manifold in representation space is that, it is the most efficient way to represent the information where linear perturbations in higher levels will still move near manifold [16,24]. This process also helps to reduce dimensionality of the input because we will need less and less manifold coordinates to span an unfolded manifold as deeper as our model. In this study we employed stacked sparse autoencoders to learn a low-dimensional non-linear feature representations for fmri pattern classification. The sparsity term and number of neurons in each layer of stacked autoencoder, are selected to lower the representation dimension without hurting the reconstruction error drastically. In order to obtain a stacked autoencoder, we employed greedy layer-wise pre-training [12, 25] where we trained one layer at a time. That is, we first train the first level autoencoder by using training data as input, and then train the second level autoencoder with the outputs from the first layer autoencoder encoding function as input and so
3 on. Note that, pre-training is performed in an unsupervised fashion. We can formalize a stacked autoencoder as follows, h L = f (L) ( f (2) (f (1) (x))), (5) where we discard decoder functions g( ) as we finish training the autoencoder, starting from the very first layer using input data. Final representation (manifold coordinates) can be obtained by the encoder function of the last layer L with the hidden representations h L. 4. CLASSIFICATION OF BRAIN STATES USING DEEP REPRESENTATIONS To improve classification performance, deep learning architectures can be fine-tuned for discrimination which is called supervised fine-tuning [25]. Final fine-tuning phase makes all the parameters {θ (i) 1 }L i=1 tuned for the supervised task at hand, by modifying representations slightly to get the category boundaries right. This can be achieved easily by unfolding the stacked autoencoder into a multi-layer neural network (initializing the weights) and adding a fully connected softmax layer on top where the number of neurons in the softmax layer is equal to the number of classes to be discriminated. It is then straight forward to optimize the network by backpropagation with gradient descent using the error derivatives of the top-most softmax layer. As expected, fine-tuning phase necessitates labelled data, but due to the pre-training phase, it does not need to discover new features, which reduces the need for a high number of labelled samples for the discriminative phase. It has also been shown that this type of backpropagation works well even if most of the data is unlabelled [13], which might often be the case for fmri pattern classification tasks. 5. EXPERIMENTS ON FMRI DATA REPRESENTATION AND CLASSIFICATION fmri data is composed of 3-dimensional brain volumes across time {t i } n i=1, where each 3D volume is formed by stacking several 2D scans (slices). Each pixel in these 2D images are actually represents the intensity of a small volume of brain tissue (voxel) at a time instant t i. A typical fmri experiment consists of several runs, where in each run the subject is exposed to some task specific stimuli {c j } S j=1 at the predefined time instants, where S is the total number of semantic classes in the experiment and n is the total length of the experiment across runs (see Figure 1). The problem arises for classification tasks here because within a large amount of samples across time, only few have assigned class labels. The rest of the samples that are not having a class label t i/cj are simply discarded. The unsupervised feature learning motivation in this study is rooted with this fact; namely, by making use of the discarded data, which might be as high as ten times more, compared to labelled samples in a typical experiment Data and Preprocessing In the current study, fmri recording was conducted during a recognition memory task. Each participant is shown a list Fig. 1. Example of a typical fmri experiment for brain state decoding. 4D fmri data consists of several volumes across time, some of which have assigned to a class labels(indicated by vertical lines in time axis) of words belonging to a specified category in the encoding phase (e.g. fruits or tools). Following a delay period where the participant solves mathematical problems, a test probe is presented and the participant executes a yes/no response indicating whether the word belongs to the current study list (e.g., see [5]). For the classification task we focused on lateral temporal cortex region having 8142 voxels. fmri data consists of 2400 time points with 240 class labels for the encoding phase and 240 class labels for the retrieval phase (for each of ten classes, 24 samples are obtained for both encoding and retrieval) Classification of Brain States The classification task we try to accomplish is to predict class labels of the samples in the retrieval phase by using samples in the encoding phase. Measurements recorded in the encoding phase are used as labelled training samples and measurements in the retrieval phase are used as test samples. The critical component in this supervised classification task is the way we incorporated the unlabelled samples. All the unlabelled samples along with labelled training samples of encoding phase are used in unsupervised feature learning with a stacked autoencoder. After learning a high level representation, labelled data is employed to either fine tune the model or directly to classify using the last layer activation of stacked autoencoder as described above Testing Procedures In order to assess the applicability of the deep learning methods on the fmri data, we initially used a small subset of measurements (100 voxels) and trained a stacked autoencoder up to two levels. Next, we employed a larger set of voxels (1024) stacked up to three levels. In both settings, we compared the results with the state of the art MVPA methods. The first method used for comparison, is the classical approach where voxel intensity values are directly fed into classifiers (indicated as MVPA). The second method is a local-linear model, where each voxel is represented in the feature space with the linear regression weights estimated using its surrounding voxels (indicated as MAD) [26]. For the unsupervised feature
4 Fig. 2. A subset of first layer filters learned using sparse autoencoder for 100 voxels in a 2D slice. Each square box above corresponds to a filter that is sensitive for a specific activation pattern of voxels. A general wedge shape activation pattern with varying intensity levels is observed along diagonals.) learning phase, we employed all measurements (2160), excluding the retrieval samples to be used for the test phase. For sparse auto-encoders, untied weights ( θ2 T ) were used in all layers as it gives better performance than tied weights in our experiments. In order to reduce dimensions in the top-most layer representations, we gradually decreased the number of neurons in each layer and reduce the β parameter that controls the importance of sparsity. For the fine-tuning phase, we attached a softmax layer with 10 units on top. For comparison with baseline MVPA methods, penultimate layer activations were fed to a k-nearest Neighbor (knn) classifier. Model parameters and hyper-parameters were empirically selected on a held-out cross validation set. For implementations of the methods mentioned above, we used liborf ( e /liborf.html) 6. RESULTS In order to qualitatively assess the validity of the unsupervised feature learning phase, we visualized the first layer filters learned for stacked autoencoders with 100 voxel experiment. The learned filters are illustrated in Figure 2. It is expected to observe reasonable activation patterns for these 100 voxels since they reside in a spatially connected space (nearest voxels in a 2D-slice). Plausible patterns should resemble an fmri alike intensity map as illustrated in Figure 2. It can be deduced that, the selected 100 voxels have various activation patterns that exhibit a wedge like pattern with various intensity levels along diagonal of learned filters. Learned filters are also very similar to actual fmri 2D-slices when we zoom in and compare with the slice in Figure 1. As the aim of unsupervised feature learning is to capture a codeword for fmri data in a lower dimensional space (manifold), we can conclude that it is promising to use the activation patterns learned in this phase to construct a feature space for the forthcoming classification task. Pattern analysis results were evaluated by considering final feature space dimensions in classifiers by comparing classification accuracy, shown in Table 1. Proposed non-linear feature mapping (NLFM) method using stacked sparse autoencoders is tested with varying depth and fine-tuning options (indicated with a * in Table 1). A shallow feature learning with a single layer autoencoder (NLFM1) is tested ini- Method 100 voxels 1024 voxels Employed Acc % Dim Acc % Dim Classifier MVPA knn MAD [26] knn NLFM softmax NLFM1* softmax NLFM1* knn NLFM knn NLFM knn Table 1. Classification performances of Non-Linear Feature Mapping (NLFM) with classical methods for fmri ML tasks (MVPA) and a local-linear feature extraction method (MAD). Feature dimensions are indicated in Dim columns. tially which gives almost similar performance with the baseline methods (MVPA,MAD), by reducing the dimension by half, which is expected as the non-linear mapping compared to the lieear ones. We further reduced the dimension by stacking more layers up to 150 from 1024 for the second experiment where we consider 1024 voxels. Classification performances gradually increase as we stack up additional layers which can be seen in Table 1 up to 59% and 72% respectively for 100 and 1024 voxels experiments. We also observed a notable performance increase with fine-tuning in our experiments. Compared to the baseline methods we observed a drastic reduction in the feature dimensions (150 compared to 1024 of MVPA and for MAD), with improving performance. Increased dimensions in the feature space is prone to over-fitting in fmri classification tasks when we consider the scantiness of labelled data. As we increase the feature dimensions the gain in the generalization performance gets stuck up to a level due to over-fitting where curse-of-dimensionality arises with linear models. The results show that multi-layer non-linear feature mapping methods are promising for high dimensional input spaces with a low number of labelled samples by leveraging unlabelled data. 7. CONCLUSION In this investigation, we modeled the neural activity using non-linear feature mapping strategies. We explored the the multiple layers of autoencoders in a deep learning framework. Stacked sparse autoencoders are used in order to learn a nonlinear feature hierarchy with using the unlabeled fmri data. The manifold assumption is further exploited by reducing dimensions as we stack up layers, resulting in a low dimensional feature space that is further used as the feature space for two fmri classification tasks. In addition to being able to efficiently reducing dimensions, results indicated an improved performance compared to the baseline methods. Future work can target application of the proposed method to whole brain data for brain decoding using 3D convolutional models. We further suggest to employing manifold tangent based classification regimes for k- Nearest Neighbor methods to further improve classification performance.
5 8. REFERENCES [1] K. A. Norman, S. M. Polyn, G. J. Detre, and J. V. Haxby, Beyond mind-reading: multi-voxel pattern analysis of fmri data. Trends in cognitive sciences, vol. 10, [2] B. Chai, D. B. Walther, D. M. Beck, and F.-F. L., Exploring Functional Connectivity of the Human Brain using Multivariate Information Analysis, Advances in neural information processing systems, [3] T. Schmah, G. Yourganov, R. S. Zemel, G. E. Hinton, S. L. Small, and S. C. Strother, Comparing classification methods for longitudinal fmri studies. Neural computation, vol. 22, [4] M. N. Coutanche, S. L. Thompson-Schill, and R. T. Schultz, Multi-voxel pattern analysis of fmri data predicts clinical symptom severity. NeuroImage, vol. 57, [5] I. Oztekin and D. Badre, Distributed Patterns of Brain Activity that Lead to Forgetting. Frontiers in human neuroscience, vol. 5, [6] T. Schmah, G. E. Hinton, R. S. Zemel, S. L. Small, and S. C. Strother, Generative versus discriminative training of rbms for classification of fmri images. in NIPS, [7] J.-D. Haynes and G. Rees, Decoding mental states from brain activity in humans. Nature reviews. Neuroscience, vol. 7, [8] L. I. Kuncheva and J. J. Rodríguez, Classifier ensembles for fmri data analysis: an experiment. Magnetic resonance imaging, vol. 28, [9] Y. Bengio, Learning Deep Architectures for AI, Foundations and Trends in Machine Learning, vol. 2, [10] I. Arel, D. C. Rose, and T. P. Karnowski, Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier], IEEE Computational Intelligence Magazine, vol. 5, [11] Y. Bengio, A. Courville, and P. Vincent, Representation learning: a review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, vol. 35, [12] Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle et al., Greedy layer-wise training of deep networks, Advances in neural information processing systems, [13] D. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent, and S. Bengio, Why Does Unsupervised Pre-training Help Deep Learning? The Journal of Machine Learning Research, vol. 11, [14] Y. Bengio, Deep learning of representations: Looking forward, CoRR, vol. abs/ , [15] Y. Bengio and Y. LeCun, Scaling learning algorithms towards ai, Large-Scale Kernel Machines, vol. 34, [16] S. Rifai, Y. Dauphin, P. Vincent, Y. Bengio, and X. Muller, The manifold tangent classifier, in NIPS, 2011, pp [17] M. Kim, G. Wu, and D. Shen, Unsupervised deep learning for hippocampus segmentation in 7.0 tesla mr images, in Machine Learning in Medical Imaging, [18] H.-I. Suk and D. Shen, Deep learning-based feature representation for ad/mci classification, in Medical Image Computing and Computer-Assisted Intervention MICCAI 2013, [19] H.-C. Shin, M. R. Orton, D. J. Collins, S. J. Doran, and M. O. Leach, Stacked autoencoders for unsupervised feature learning and multiple organ detection in a pilot study using 4d patient data, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 35, [20] T. Brosch and R. Tam, Manifold learning of brain mris by deep learning, in Medical Image Computing and Computer-Assisted Intervention MICCAI 2013, [21] K. Kavukcuoglu, M. Ranzato, R. Fergus, and Y. Le- Cun, Learning invariant features through topographic filter maps, in Computer Vision and Pattern Recognition,CVPR. IEEE Conference on, 2009, pp [22] S. Rifai, P. Vincent, X. Muller, X. Glorot, and Y. Bengio, Contractive auto-encoders: Explicit invariance during feature extraction, in Proceedings of the 28th International Conference on Machine Learning (ICML-11), [23] P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, Extracting and composing robust features with denoising autoencoders, in Proceedings of the 25th international conference on Machine learning - ICML 08, [24] Y. Bengio, E. Thibodeau-Laufer, and J. Yosinski, Deep generative stochastic networks trainable by backprop, CoRR, vol. abs/ , [25] G. E. Hinton, S. Osindero, and Y.-W. Teh, A fast learning algorithm for deep belief nets, Neural computation, vol. 18, [26] M. Ozay, I. Öztekin, U. Öztekin, and F. T. Y. Vural, Mesh Learning for Classifying Cognitive Processes, Arxiv: , 2012.
Introduction to Deep Learning
ENEE698A : Machine Learning Seminar Introduction to Deep Learning Raviteja Vemulapalli Image credit: [LeCun 1998] Resources Unsupervised feature learning and deep learning (UFLDL) tutorial (http://ufldl.stanford.edu/wiki/index.php/ufldl_tutorial)
More informationAutoencoders, denoising autoencoders, and learning deep networks
4 th CiFAR Summer School on Learning and Vision in Biology and Engineering Toronto, August 5-9 2008 Autoencoders, denoising autoencoders, and learning deep networks Part II joint work with Hugo Larochelle,
More informationNeural Networks: promises of current research
April 2008 www.apstat.com Current research on deep architectures A few labs are currently researching deep neural network training: Geoffrey Hinton s lab at U.Toronto Yann LeCun s lab at NYU Our LISA lab
More informationNeural Networks for Machine Learning. Lecture 15a From Principal Components Analysis to Autoencoders
Neural Networks for Machine Learning Lecture 15a From Principal Components Analysis to Autoencoders Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed Principal Components
More informationDepth Image Dimension Reduction Using Deep Belief Networks
Depth Image Dimension Reduction Using Deep Belief Networks Isma Hadji* and Akshay Jain** Department of Electrical and Computer Engineering University of Missouri 19 Eng. Building West, Columbia, MO, 65211
More informationA Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images Marc Aurelio Ranzato Yann LeCun Courant Institute of Mathematical Sciences New York University - New York, NY 10003 Abstract
More informationAdvanced Introduction to Machine Learning, CMU-10715
Advanced Introduction to Machine Learning, CMU-10715 Deep Learning Barnabás Póczos, Sept 17 Credits Many of the pictures, results, and other materials are taken from: Ruslan Salakhutdinov Joshua Bengio
More informationA Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images Marc Aurelio Ranzato Yann LeCun Courant Institute of Mathematical Sciences New York University - New York, NY 10003 Abstract
More informationCOMP 551 Applied Machine Learning Lecture 16: Deep Learning
COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all
More informationDeep Learning. Deep Learning. Practical Application Automatically Adding Sounds To Silent Movies
http://blog.csdn.net/zouxy09/article/details/8775360 Automatic Colorization of Black and White Images Automatically Adding Sounds To Silent Movies Traditionally this was done by hand with human effort
More informationAkarsh Pokkunuru EECS Department Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
Akarsh Pokkunuru EECS Department 03-16-2017 Contractive Auto-Encoders: Explicit Invariance During Feature Extraction 1 AGENDA Introduction to Auto-encoders Types of Auto-encoders Analysis of different
More informationStacked Denoising Autoencoders for Face Pose Normalization
Stacked Denoising Autoencoders for Face Pose Normalization Yoonseop Kang 1, Kang-Tae Lee 2,JihyunEun 2, Sung Eun Park 2 and Seungjin Choi 1 1 Department of Computer Science and Engineering Pohang University
More informationMachine Learning. Deep Learning. Eric Xing (and Pengtao Xie) , Fall Lecture 8, October 6, Eric CMU,
Machine Learning 10-701, Fall 2015 Deep Learning Eric Xing (and Pengtao Xie) Lecture 8, October 6, 2015 Eric Xing @ CMU, 2015 1 A perennial challenge in computer vision: feature engineering SIFT Spin image
More informationDeep Learning. Deep Learning provided breakthrough results in speech recognition and image classification. Why?
Data Mining Deep Learning Deep Learning provided breakthrough results in speech recognition and image classification. Why? Because Speech recognition and image classification are two basic examples of
More informationDEEP LEARNING TO DIVERSIFY BELIEF NETWORKS FOR REMOTE SENSING IMAGE CLASSIFICATION
DEEP LEARNING TO DIVERSIFY BELIEF NETWORKS FOR REMOTE SENSING IMAGE CLASSIFICATION S.Dhanalakshmi #1 #PG Scholar, Department of Computer Science, Dr.Sivanthi Aditanar college of Engineering, Tiruchendur
More informationEnergy Based Models, Restricted Boltzmann Machines and Deep Networks. Jesse Eickholt
Energy Based Models, Restricted Boltzmann Machines and Deep Networks Jesse Eickholt ???? Who s heard of Energy Based Models (EBMs) Restricted Boltzmann Machines (RBMs) Deep Belief Networks Auto-encoders
More informationExtracting and Composing Robust Features with Denoising Autoencoders
Presenter: Alexander Truong March 16, 2017 Extracting and Composing Robust Features with Denoising Autoencoders Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre-Antoine Manzagol 1 Outline Introduction
More informationSupervised Learning for Image Segmentation
Supervised Learning for Image Segmentation Raphael Meier 06.10.2016 Raphael Meier MIA 2016 06.10.2016 1 / 52 References A. Ng, Machine Learning lecture, Stanford University. A. Criminisi, J. Shotton, E.
More information3D Object Recognition with Deep Belief Nets
3D Object Recognition with Deep Belief Nets Vinod Nair and Geoffrey E. Hinton Department of Computer Science, University of Toronto 10 King s College Road, Toronto, M5S 3G5 Canada {vnair,hinton}@cs.toronto.edu
More informationContractive Auto-Encoders: Explicit Invariance During Feature Extraction
: Explicit Invariance During Feature Extraction Salah Rifai (1) Pascal Vincent (1) Xavier Muller (1) Xavier Glorot (1) Yoshua Bengio (1) (1) Dept. IRO, Université de Montréal. Montréal (QC), H3C 3J7, Canada
More informationAn Empirical Evaluation of Deep Architectures on Problems with Many Factors of Variation
An Empirical Evaluation of Deep Architectures on Problems with Many Factors of Variation Hugo Larochelle, Dumitru Erhan, Aaron Courville, James Bergstra, and Yoshua Bengio Université de Montréal 13/06/2007
More informationFacial Expression Classification with Random Filters Feature Extraction
Facial Expression Classification with Random Filters Feature Extraction Mengye Ren Facial Monkey mren@cs.toronto.edu Zhi Hao Luo It s Me lzh@cs.toronto.edu I. ABSTRACT In our work, we attempted to tackle
More informationMachine Learning. The Breadth of ML Neural Networks & Deep Learning. Marc Toussaint. Duy Nguyen-Tuong. University of Stuttgart
Machine Learning The Breadth of ML Neural Networks & Deep Learning Marc Toussaint University of Stuttgart Duy Nguyen-Tuong Bosch Center for Artificial Intelligence Summer 2017 Neural Networks Consider
More informationMachine Learning. MGS Lecture 3: Deep Learning
Dr Michel F. Valstar http://cs.nott.ac.uk/~mfv/ Machine Learning MGS Lecture 3: Deep Learning Dr Michel F. Valstar http://cs.nott.ac.uk/~mfv/ WHAT IS DEEP LEARNING? Shallow network: Only one hidden layer
More informationDeep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks
Deep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks Si Chen The George Washington University sichen@gwmail.gwu.edu Meera Hahn Emory University mhahn7@emory.edu Mentor: Afshin
More informationTransfer Learning Using Rotated Image Data to Improve Deep Neural Network Performance
Transfer Learning Using Rotated Image Data to Improve Deep Neural Network Performance Telmo Amaral¹, Luís M. Silva¹², Luís A. Alexandre³, Chetak Kandaswamy¹, Joaquim Marques de Sá¹ 4, and Jorge M. Santos¹
More informationNeural Network Neurons
Neural Networks Neural Network Neurons 1 Receives n inputs (plus a bias term) Multiplies each input by its weight Applies activation function to the sum of results Outputs result Activation Functions Given
More informationLearning Two-Layer Contractive Encodings
In Proceedings of International Conference on Artificial Neural Networks (ICANN), pp. 620-628, September 202. Learning Two-Layer Contractive Encodings Hannes Schulz and Sven Behnke Rheinische Friedrich-Wilhelms-Universität
More informationDEEP LEARNING REVIEW. Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature Presented by Divya Chitimalla
DEEP LEARNING REVIEW Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature 2015 -Presented by Divya Chitimalla What is deep learning Deep learning allows computational models that are composed of multiple
More informationMachine Learning 13. week
Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of
More informationDeep Similarity Learning for Multimodal Medical Images
Deep Similarity Learning for Multimodal Medical Images Xi Cheng, Li Zhang, and Yefeng Zheng Siemens Corporation, Corporate Technology, Princeton, NJ, USA Abstract. An effective similarity measure for multi-modal
More informationStatic Gesture Recognition with Restricted Boltzmann Machines
Static Gesture Recognition with Restricted Boltzmann Machines Peter O Donovan Department of Computer Science, University of Toronto 6 Kings College Rd, M5S 3G4, Canada odonovan@dgp.toronto.edu Abstract
More informationDEEP learning algorithms have been the object of much
1 Is Joint Training Better for Deep Auto-Encoders? Yingbo Zhou, Devansh Arpit, Ifeoma Nwogu, Venu Govindaraju arxiv:1405.1380v4 [stat.ml] 15 Jun 2015 Abstract Traditionally, when generative models of data
More informationDeep Learning. Vladimir Golkov Technical University of Munich Computer Vision Group
Deep Learning Vladimir Golkov Technical University of Munich Computer Vision Group 1D Input, 1D Output target input 2 2D Input, 1D Output: Data Distribution Complexity Imagine many dimensions (data occupies
More informationNeural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer
More informationTo be Bernoulli or to be Gaussian, for a Restricted Boltzmann Machine
2014 22nd International Conference on Pattern Recognition To be Bernoulli or to be Gaussian, for a Restricted Boltzmann Machine Takayoshi Yamashita, Masayuki Tanaka, Eiji Yoshida, Yuji Yamauchi and Hironobu
More informationNeural Networks for unsupervised learning From Principal Components Analysis to Autoencoders to semantic hashing
Neural Networks for unsupervised learning From Principal Components Analysis to Autoencoders to semantic hashing feature 3 PC 3 Beate Sick Many slides are taken form Hinton s great lecture on NN: https://www.coursera.org/course/neuralnets
More informationThe Anatomical Equivalence Class Formulation and its Application to Shape-based Computational Neuroanatomy
The Anatomical Equivalence Class Formulation and its Application to Shape-based Computational Neuroanatomy Sokratis K. Makrogiannis, PhD From post-doctoral research at SBIA lab, Department of Radiology,
More informationDeep Learning Basic Lecture - Complex Systems & Artificial Intelligence 2017/18 (VO) Asan Agibetov, PhD.
Deep Learning 861.061 Basic Lecture - Complex Systems & Artificial Intelligence 2017/18 (VO) Asan Agibetov, PhD asan.agibetov@meduniwien.ac.at Medical University of Vienna Center for Medical Statistics,
More informationNovel Lossy Compression Algorithms with Stacked Autoencoders
Novel Lossy Compression Algorithms with Stacked Autoencoders Anand Atreya and Daniel O Shea {aatreya, djoshea}@stanford.edu 11 December 2009 1. Introduction 1.1. Lossy compression Lossy compression is
More informationNeural Networks and Deep Learning
Neural Networks and Deep Learning Example Learning Problem Example Learning Problem Celebrity Faces in the Wild Machine Learning Pipeline Raw data Feature extract. Feature computation Inference: prediction,
More informationCharacter Recognition Using Convolutional Neural Networks
Character Recognition Using Convolutional Neural Networks David Bouchain Seminar Statistical Learning Theory University of Ulm, Germany Institute for Neural Information Processing Winter 2006/2007 Abstract
More informationCMU Lecture 18: Deep learning and Vision: Convolutional neural networks. Teacher: Gianni A. Di Caro
CMU 15-781 Lecture 18: Deep learning and Vision: Convolutional neural networks Teacher: Gianni A. Di Caro DEEP, SHALLOW, CONNECTED, SPARSE? Fully connected multi-layer feed-forward perceptrons: More powerful
More informationDay 3 Lecture 1. Unsupervised Learning
Day 3 Lecture 1 Unsupervised Learning Semi-supervised and transfer learning Myth: you can t do deep learning unless you have a million labelled examples for your problem. Reality You can learn useful representations
More informationDeep Generative Models Variational Autoencoders
Deep Generative Models Variational Autoencoders Sudeshna Sarkar 5 April 2017 Generative Nets Generative models that represent probability distributions over multiple variables in some way. Directed Generative
More informationDeep Learning for Computer Vision II
IIIT Hyderabad Deep Learning for Computer Vision II C. V. Jawahar Paradigm Shift Feature Extraction (SIFT, HoG, ) Part Models / Encoding Classifier Sparrow Feature Learning Classifier Sparrow L 1 L 2 L
More informationToward a rigorous statistical framework for brain mapping
Toward a rigorous statistical framework for brain mapping Bertrand Thirion, bertrand.thirion@inria.fr 1 Cognitive neuroscience How are cognitive activities affected or controlled by neural circuits in
More informationDoes the Brain do Inverse Graphics?
Does the Brain do Inverse Graphics? Geoffrey Hinton, Alex Krizhevsky, Navdeep Jaitly, Tijmen Tieleman & Yichuan Tang Department of Computer Science University of Toronto The representation used by the
More informationFactor Topographic Latent Source Analysis: Factor Analysis for Brain Images
Factor Topographic Latent Source Analysis: Factor Analysis for Brain Images Jeremy R. Manning 1,2, Samuel J. Gershman 3, Kenneth A. Norman 1,3, and David M. Blei 2 1 Princeton Neuroscience Institute, Princeton
More informationIEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS, VOL. 3, NO. 4, DECEMBER
IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS, VOL. 3, NO. 4, DECEMBER 2017 683 A New Representation of fmri Signal by a Set of Local Meshes for Brain Decoding Itir Onal, Student
More informationEffectiveness of Sparse Features: An Application of Sparse PCA
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
More informationFeature Extraction and Learning for RSSI based Indoor Device Localization
Feature Extraction and Learning for RSSI based Indoor Device Localization Stavros Timotheatos, Grigorios Tsagkatakis, Panagiotis Tsakalides and Panos Trahanias Institute of Computer Science Foundation
More informationConvolutional Neural Networks
Lecturer: Barnabas Poczos Introduction to Machine Learning (Lecture Notes) Convolutional Neural Networks Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.
More informationDoes the Brain do Inverse Graphics?
Does the Brain do Inverse Graphics? Geoffrey Hinton, Alex Krizhevsky, Navdeep Jaitly, Tijmen Tieleman & Yichuan Tang Department of Computer Science University of Toronto How to learn many layers of features
More informationComputer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning for Object Categorization 14.01.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period
More informationUnsupervised Learning of Spatiotemporally Coherent Metrics
Unsupervised Learning of Spatiotemporally Coherent Metrics Ross Goroshin, Joan Bruna, Jonathan Tompson, David Eigen, Yann LeCun arxiv 2015. Presented by Jackie Chu Contributions Insight between slow feature
More informationOutlier detection using autoencoders
Outlier detection using autoencoders August 19, 2016 Author: Olga Lyudchik Supervisors: Dr. Jean-Roch Vlimant Dr. Maurizio Pierini CERN Non Member State Summer Student Report 2016 Abstract Outlier detection
More informationAutoencoders. Stephen Scott. Introduction. Basic Idea. Stacked AE. Denoising AE. Sparse AE. Contractive AE. Variational AE GAN.
Stacked Denoising Sparse Variational (Adapted from Paul Quint and Ian Goodfellow) Stacked Denoising Sparse Variational Autoencoding is training a network to replicate its input to its output Applications:
More informationAutoencoder. Representation learning (related to dictionary learning) Both the input and the output are x
Deep Learning 4 Autoencoder, Attention (spatial transformer), Multi-modal learning, Neural Turing Machine, Memory Networks, Generative Adversarial Net Jian Li IIIS, Tsinghua Autoencoder Autoencoder Unsupervised
More informationUnsupervised learning in Vision
Chapter 7 Unsupervised learning in Vision The fields of Computer Vision and Machine Learning complement each other in a very natural way: the aim of the former is to extract useful information from visual
More informationDeep Learning. Volker Tresp Summer 2014
Deep Learning Volker Tresp Summer 2014 1 Neural Network Winter and Revival While Machine Learning was flourishing, there was a Neural Network winter (late 1990 s until late 2000 s) Around 2010 there
More informationSPE MS. Abstract. Introduction. Autoencoders
SPE-174015-MS Autoencoder-derived Features as Inputs to Classification Algorithms for Predicting Well Failures Jeremy Liu, ISI USC, Ayush Jaiswal, USC, Ke-Thia Yao, ISI USC, Cauligi S.Raghavendra, USC
More informationResearch on Pruning Convolutional Neural Network, Autoencoder and Capsule Network
Research on Pruning Convolutional Neural Network, Autoencoder and Capsule Network Tianyu Wang Australia National University, Colledge of Engineering and Computer Science u@anu.edu.au Abstract. Some tasks,
More informationSIIM 2017 Scientific Session Analytics & Deep Learning Part 2 Friday, June 2 8:00 am 9:30 am
SIIM 2017 Scientific Session Analytics & Deep Learning Part 2 Friday, June 2 8:00 am 9:30 am Performance of Deep Convolutional Neural Networks for Classification of Acute Territorial Infarct on Brain MRI:
More informationSparse Models in Image Understanding And Computer Vision
Sparse Models in Image Understanding And Computer Vision Jayaraman J. Thiagarajan Arizona State University Collaborators Prof. Andreas Spanias Karthikeyan Natesan Ramamurthy Sparsity Sparsity of a vector
More informationNeural Network and Deep Learning. Donglin Zeng, Department of Biostatistics, University of North Carolina
Neural Network and Deep Learning Early history of deep learning Deep learning dates back to 1940s: known as cybernetics in the 1940s-60s, connectionism in the 1980s-90s, and under the current name starting
More informationLimited Generalization Capabilities of Autoencoders with Logistic Regression on Training Sets of Small Sizes
Limited Generalization Capabilities of Autoencoders with Logistic Regression on Training Sets of Small Sizes Alexey Potapov 1,2, Vita Batishcheva 2, and Maxim Peterson 1 1 St. Petersburg ational Research
More informationRepresentational similarity analysis. Dr Ian Charest,
Representational similarity analysis Dr Ian Charest, Edinburgh, April 219 A space for neuroimaging studies Pattern across subjects Cross-subject correlation studies (e.g. social neuroscience) Raizada et
More informationNeural Networks. Single-layer neural network. CSE 446: Machine Learning Emily Fox University of Washington March 10, /10/2017
3/0/207 Neural Networks Emily Fox University of Washington March 0, 207 Slides adapted from Ali Farhadi (via Carlos Guestrin and Luke Zettlemoyer) Single-layer neural network 3/0/207 Perceptron as a neural
More informationSEMANTIC COMPUTING. Lecture 8: Introduction to Deep Learning. TU Dresden, 7 December Dagmar Gromann International Center For Computational Logic
SEMANTIC COMPUTING Lecture 8: Introduction to Deep Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 7 December 2018 Overview Introduction Deep Learning General Neural Networks
More informationDeep Learning With Noise
Deep Learning With Noise Yixin Luo Computer Science Department Carnegie Mellon University yixinluo@cs.cmu.edu Fan Yang Department of Mathematical Sciences Carnegie Mellon University fanyang1@andrew.cmu.edu
More informationMULTIVARIATE ANALYSES WITH fmri DATA
MULTIVARIATE ANALYSES WITH fmri DATA Sudhir Shankar Raman Translational Neuromodeling Unit (TNU) Institute for Biomedical Engineering University of Zurich & ETH Zurich Motivation Modelling Concepts Learning
More informationMulti-Glance Attention Models For Image Classification
Multi-Glance Attention Models For Image Classification Chinmay Duvedi Stanford University Stanford, CA cduvedi@stanford.edu Pararth Shah Stanford University Stanford, CA pararth@stanford.edu Abstract We
More informationChallenges motivating deep learning. Sargur N. Srihari
Challenges motivating deep learning Sargur N. srihari@cedar.buffalo.edu 1 Topics In Machine Learning Basics 1. Learning Algorithms 2. Capacity, Overfitting and Underfitting 3. Hyperparameters and Validation
More informationarxiv: v1 [cs.ne] 28 Nov 2017
Block Neural Network Avoids Catastrophic Forgetting When Learning Multiple Task arxiv:1711.10204v1 [cs.ne] 28 Nov 2017 Guglielmo Montone montone.guglielmo@gmail.com Alexander V. Terekhov avterekhov@gmail.com
More informationA Deep Learning primer
A Deep Learning primer Riccardo Zanella r.zanella@cineca.it SuperComputing Applications and Innovation Department 1/21 Table of Contents Deep Learning: a review Representation Learning methods DL Applications
More informationDynamic Routing Between Capsules
Report Explainable Machine Learning Dynamic Routing Between Capsules Author: Michael Dorkenwald Supervisor: Dr. Ullrich Köthe 28. Juni 2018 Inhaltsverzeichnis 1 Introduction 2 2 Motivation 2 3 CapusleNet
More informationVoxel selection algorithms for fmri
Voxel selection algorithms for fmri Henryk Blasinski December 14, 2012 1 Introduction Functional Magnetic Resonance Imaging (fmri) is a technique to measure and image the Blood- Oxygen Level Dependent
More informationLearning-based Neuroimage Registration
Learning-based Neuroimage Registration Leonid Teverovskiy and Yanxi Liu 1 October 2004 CMU-CALD-04-108, CMU-RI-TR-04-59 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Abstract
More informationPouya Kousha Fall 2018 CSE 5194 Prof. DK Panda
Pouya Kousha Fall 2018 CSE 5194 Prof. DK Panda 1 Observe novel applicability of DL techniques in Big Data Analytics. Applications of DL techniques for common Big Data Analytics problems. Semantic indexing
More informationLearning Discrete Representations via Information Maximizing Self-Augmented Training
A. Relation to Denoising and Contractive Auto-encoders Our method is related to denoising auto-encoders (Vincent et al., 2008). Auto-encoders maximize a lower bound of mutual information (Cover & Thomas,
More informationA Fast Learning Algorithm for Deep Belief Nets
A Fast Learning Algorithm for Deep Belief Nets Geoffrey E. Hinton, Simon Osindero Department of Computer Science University of Toronto, Toronto, Canada Yee-Whye Teh Department of Computer Science National
More informationCSC321: Neural Networks. Lecture 13: Learning without a teacher: Autoencoders and Principal Components Analysis. Geoffrey Hinton
CSC321: Neural Networks Lecture 13: Learning without a teacher: Autoencoders and Principal Components Analysis Geoffrey Hinton Three problems with backpropagation Where does the supervision come from?
More informationSu et al. Shape Descriptors - III
Su et al. Shape Descriptors - III Siddhartha Chaudhuri http://www.cse.iitb.ac.in/~cs749 Funkhouser; Feng, Liu, Gong Recap Global A shape descriptor is a set of numbers that describes a shape in a way that
More informationDeep Learning. Architecture Design for. Sargur N. Srihari
Architecture Design for Deep Learning Sargur N. srihari@cedar.buffalo.edu 1 Topics Overview 1. Example: Learning XOR 2. Gradient-Based Learning 3. Hidden Units 4. Architecture Design 5. Backpropagation
More informationEfficient Algorithms may not be those we think
Efficient Algorithms may not be those we think Yann LeCun, Computational and Biological Learning Lab The Courant Institute of Mathematical Sciences New York University http://yann.lecun.com http://www.cs.nyu.edu/~yann
More informationSeminars in Artifiial Intelligenie and Robotiis
Seminars in Artifiial Intelligenie and Robotiis Computer Vision for Intelligent Robotiis Basiis and hints on CNNs Alberto Pretto What is a neural network? We start from the frst type of artifcal neuron,
More informationConvolutional Neural Networks. Computer Vision Jia-Bin Huang, Virginia Tech
Convolutional Neural Networks Computer Vision Jia-Bin Huang, Virginia Tech Today s class Overview Convolutional Neural Network (CNN) Training CNN Understanding and Visualizing CNN Image Categorization:
More informationImage Denoising and Inpainting with Deep Neural Networks
Image Denoising and Inpainting with Deep Neural Networks Junyuan Xie, Linli Xu, Enhong Chen School of Computer Science and Technology University of Science and Technology of China eric.jy.xie@gmail.com,
More informationAdditive Manufacturing Defect Detection using Neural Networks
Additive Manufacturing Defect Detection using Neural Networks James Ferguson Department of Electrical Engineering and Computer Science University of Tennessee Knoxville Knoxville, Tennessee 37996 Jfergu35@vols.utk.edu
More informationDeep Learning for Generic Object Recognition
Deep Learning for Generic Object Recognition, Computational and Biological Learning Lab The Courant Institute of Mathematical Sciences New York University Collaborators: Marc'Aurelio Ranzato, Fu Jie Huang,
More informationDivya Ramesh December 8, 2014
EE 660 Machine Learning from Signals: Foundations and Methods Project Title: Analysis of Unsupervised Feature Learning Methods for Image Classification Divya Ramesh (dramesh@usc.edu) 5924-4440-58 December
More informationVisual object classification by sparse convolutional neural networks
Visual object classification by sparse convolutional neural networks Alexander Gepperth 1 1- Ruhr-Universität Bochum - Institute for Neural Dynamics Universitätsstraße 150, 44801 Bochum - Germany Abstract.
More informationClassification of Subject Motion for Improved Reconstruction of Dynamic Magnetic Resonance Imaging
1 CS 9 Final Project Classification of Subject Motion for Improved Reconstruction of Dynamic Magnetic Resonance Imaging Feiyu Chen Department of Electrical Engineering ABSTRACT Subject motion is a significant
More informationStudy of Residual Networks for Image Recognition
Study of Residual Networks for Image Recognition Mohammad Sadegh Ebrahimi Stanford University sadegh@stanford.edu Hossein Karkeh Abadi Stanford University hosseink@stanford.edu Abstract Deep neural networks
More informationDeep Learning on Graphs
Deep Learning on Graphs with Graph Convolutional Networks Hidden layer Hidden layer Input Output ReLU ReLU, 22 March 2017 joint work with Max Welling (University of Amsterdam) BDL Workshop @ NIPS 2016
More informationManifold Learning of Brain MRIs by Deep Learning
Manifold Learning of Brain MRIs by Deep Learning Tom Brosch 1,3 and Roger Tam 2,3 for the Alzheimer s Disease Neuroimaging Initiative 1 Department of Electrical and Computer Engineering 2 Department of
More informationEmotion Detection using Deep Belief Networks
Emotion Detection using Deep Belief Networks Kevin Terusaki and Vince Stigliani May 9, 2014 Abstract In this paper, we explore the exciting new field of deep learning. Recent discoveries have made it possible
More informationCPSC340. State-of-the-art Neural Networks. Nando de Freitas November, 2012 University of British Columbia
CPSC340 State-of-the-art Neural Networks Nando de Freitas November, 2012 University of British Columbia Outline of the lecture This lecture provides an overview of two state-of-the-art neural networks:
More informationMulti-Task Learning of Facial Landmarks and Expression
Multi-Task Learning of Facial Landmarks and Expression Terrance Devries 1, Kumar Biswaranjan 2, and Graham W. Taylor 1 1 School of Engineering, University of Guelph, Guelph, Canada N1G 2W1 2 Department
More information