Multiview Feature Learning
|
|
|
- Mervyn Paul
- 5 years ago
- Views:
Transcription
1 Multiview Feature Learning Roland Memisevic Frankfurt, Montreal Tutorial at IPAM 2012 Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
2 Outline 1 Introduction Feature Learning Correspondence in Computer Vision Multiview feature learning 2 Learning relational features Encoding relations Learning 3 Factorization, eigen-spaces and complex cells Factorization Eigen-spaces, energy models, complex cells 4 Applications and extensions Applications and extensions Conclusions Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
3 Outline 1 Introduction Feature Learning Correspondence in Computer Vision Multiview feature learning 2 Learning relational features Encoding relations Learning 3 Factorization, eigen-spaces and complex cells Factorization Eigen-spaces, energy models, complex cells 4 Applications and extensions Applications and extensions Conclusions Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
4 What this is about Extend feature learning to model relations. mapping units, bi-linear models, energy-models, complex cells, spatio-temporal features, covariance features, bi-linear classification, quadrature features, gated Boltzmann machine, mcrbm,... Feature learning beyond object recognition Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
5 What this is about Extend feature learning to model relations. mapping units, bi-linear models, energy-models, complex cells, spatio-temporal features, covariance features, bi-linear classification, quadrature features, gated Boltzmann machine, mcrbm,... Feature learning beyond object recognition Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
6 Local features for recognition Object recognition started to work very well. The main reason is the use of local features. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
7 Local features for recognition Object recognition started to work very well. The main reason is the use of local features. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
8 Bag-Of-Features Bag-Of-Features 1 Find interest points. 2 Crop patches around interest points. 3 Represent each patch with a sparse local descriptor ( features ). 4 Add all local descriptors to obtain a global descriptor for the image. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
9 Bag-Of-Features Bag-Of-Features 1 Find interest points. 2 Crop patches around interest points. 3 Represent each patch with a sparse local descriptor ( features ). 4 Add all local descriptors to obtain a global descriptor for the image. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
10 Bag-Of-Features f 1 f n Bag-Of-Features 1 Find interest points. 2 Crop patches around interest points. 3 Represent each patch with a sparse local descriptor ( features ). 4 Add all local descriptors to obtain a global descriptor for the image. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
11 Bag-Of-Features f 1 1 f M 1 f 1 n f M n Bag-Of-Features 1 Find interest points. 2 Crop patches around interest points. 3 Represent each patch with a sparse local descriptor ( features ). 4 Add all local descriptors to obtain a global descriptor for the image. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
12 Bag-Of-Features f 1 1 f M 1 f 1 n f M n Bag-Of-Features 1 Find interest points. 2 Crop patches around interest points. 3 Represent each patch with a sparse local descriptor ( features ). 4 Add all local descriptors to obtain a global descriptor for the image. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
13 Convolutional Convolutional 1 Crop patches along a regular grid (dense or not). 2 Represent each patch with a local descriptor. 3 Concatenate all descriptors in a very large vector. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
14 Convolutional Convolutional 1 Crop patches along a regular grid (dense or not). 2 Represent each patch with a local descriptor. 3 Concatenate all descriptors in a very large vector. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
15 Convolutional Convolutional 1 Crop patches along a regular grid (dense or not). 2 Represent each patch with a local descriptor. 3 Concatenate all descriptors in a very large vector. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
16 Classification f 2? cathedral high-rise f 1 After computing representations, use logistic regression, SVM, NN,... There are various extensions, like fancy pooling, etc. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
17 Extracting local features f 1 f n How to extract local features. Engineer them. SIFT, HOG, LBP, etc. Learn them from image data deep learning Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
18 Extracting local features f 1 f n How to extract local features. Engineer them. SIFT, HOG, LBP, etc. Learn them from image data deep learning Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
19 Extracting local features f 1 f n How to extract local features. Engineer them. SIFT, HOG, LBP, etc. Learn them from image data deep learning Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
20 Feature learning z z k z(y) = sigmoid(w T y) y(z) = W z w jk y y j Feature learning W = arg min W y α y ( z ( y α)) 2 α Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
21 Feature learning models z z k w jk y j p(y j z) = sigmoid ( ) w jk z k p(z k y) = sigmoid ( ) w jk y j k j Restricted Boltzmann machine (RBM) p(y, z) = 1 Z exp ( jk w jky j z k ) y Learning: Maximum likelihood/contrastive divergence. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
22 Feature learning models ŷ ŷj w kj z k = sigmoid ( ) a jk y j j zk y j = k w jk z k a jk Autoencoder Add inference parameters. yj y Learning: Minimize reconstruction error. Add a sparsity penalty or corrupt inputs during training (Vincent et al., 2008). Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
23 Feature learning models z zk w jk y j = w jk z k k yj Independent Components Analysis (ICA) y Learning: Make responses sparse, while keeping filters sensible min W T y 1 W s.t. W T W = I Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
24 Feature Learning Works (CIFAR) (NORB) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
25 Manifold perspective f(z) y g(y) z Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
26 Outline 1 Introduction Feature Learning Correspondence in Computer Vision Multiview feature learning 2 Learning relational features Encoding relations Learning 3 Factorization, eigen-spaces and complex cells Factorization Eigen-spaces, energy models, complex cells 4 Applications and extensions Applications and extensions Conclusions Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
27 Beyond object recognition Can we do more with Feature Learning than recognize things? Brains can do much more than recognize objects. Many vision tasks go beyond object recognition. In surprisingly many of them, the relationship between images carries the relevant information. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
28 Beyond object recognition Can we do more with Feature Learning than recognize things? Brains can do much more than recognize objects. Many vision tasks go beyond object recognition. In surprisingly many of them, the relationship between images carries the relevant information. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
29 Correspondences in Computer Vision Correspondence is one of the most ubiquitous problems in Computer Vision. Some correspondence tasks in Vision Tracking Stereo Geometry Optical Flow Invariant Recognition Odometry Action Recognition Contours, Within-image structure Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
30 Correspondences in Computer Vision Correspondence is one of the most ubiquitous problems in Computer Vision. Some correspondence tasks in Vision Tracking Stereo Geometry Optical Flow Invariant Recognition Odometry Action Recognition Contours, Within-image structure Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
31 Correspondences in Computer Vision Correspondence is one of the most ubiquitous problems in Computer Vision. Some correspondence tasks in Vision Tracking Stereo Geometry Optical Flow Invariant Recognition Odometry Action Recognition Contours, Within-image structure Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
32 Correspondences in Computer Vision Correspondence is one of the most ubiquitous problems in Computer Vision. Some correspondence tasks in Vision Tracking Stereo Geometry Optical Flow Invariant Recognition Odometry Action Recognition Contours, Within-image structure Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
33 Correspondences in Computer Vision Correspondence is one of the most ubiquitous problems in Computer Vision. Some correspondence tasks in Vision Tracking Stereo Geometry Optical Flow Invariant Recognition Odometry Action Recognition Contours, Within-image structure Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
34 Correspondences in Computer Vision Correspondence is one of the most ubiquitous problems in Computer Vision. Some correspondence tasks in Vision Tracking Stereo Geometry Optical Flow Invariant Recognition Odometry Action Recognition Contours, Within-image structure Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
35 Correspondences in Computer Vision Correspondence is one of the most ubiquitous problems in Computer Vision. Some correspondence tasks in Vision Tracking Stereo Geometry Optical Flow Invariant Recognition Odometry Action Recognition Contours, Within-image structure Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
36 Correspondences in Computer Vision Correspondence is one of the most ubiquitous problems in Computer Vision. Some correspondence tasks in Vision Tracking Stereo Geometry Optical Flow Invariant Recognition Odometry Action Recognition Contours, Within-image structure Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
37 Correspondences in Computer Vision Correspondence is one of the most ubiquitous problems in Computer Vision. Some correspondence tasks in Vision Tracking Stereo Geometry Optical Flow Invariant Recognition Odometry Action Recognition Contours, Within-image structure Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
38 Heider and Simmel Adding frames is not just about adding proportionally more information. The relationships between frames contain additional information, that is not present in any single frame. See Heider and Simmel, 1944: Any single frame shows a bunch of geometric figures. The motions reveal the story. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
39 Random dot stereograms You can see objects even when images contain no features. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
40 Outline 1 Introduction Feature Learning Correspondence in Computer Vision Multiview feature learning 2 Learning relational features Encoding relations Learning 3 Factorization, eigen-spaces and complex cells Factorization Eigen-spaces, energy models, complex cells 4 Applications and extensions Applications and extensions Conclusions Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
41 Learning features to model correspondences If correspondences matter in vision, can we learn them? z? x y Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
42 Learning features to model correspondences We can, if we let latent variables act like gates, that dynamically change the connections between fellow variables. x i z k y j Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
43 Learning features to model correspondences Learning and inference (slightly) different from learning without. We can set things up, such that inference is almost unchanged. Yet, the meaning of the latent variables will be entirely different. x i z k y j Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
44 Learning features to model correspondences Multiplicative interactions allow hidden variables to blend in a whole sub -network. This leads to a qualitatively quite different behaviour from the common, bi-partite feature learning models. x i z k y j Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
45 Multiplicative interactions Brief history of gating Mapping units (Hinton; 1981), dynamic mappings (v.d. Malsburg; 1981) Binocular+Motion Energy models (Adelson, Bergen; 1985), (Ozhawa, DeAngelis, Freeman; 1990), (Fleet et al., 1994) Higher-order neural nets, Sigma-Pi-units Routing circuits (Olshausen; 1994) Bi-linear models (Tenenbaum, Freeman; 2000), (Grimes, Rao; 2005), (Olshausen; 2007) Subspace SOM (Kohonen, 1996) ISA, topographic ICA (Hyvarinen, Hoyer; 2000), (Karklin, Lewicki; 2003): Higher-order within image structure (2006 ) GBM, mcrbm, GAE, convisa, applications... Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
46 Multiplicative interactions Brief history of gating Mapping units (Hinton; 1981), dynamic mappings (v.d. Malsburg; 1981) Binocular+Motion Energy models (Adelson, Bergen; 1985), (Ozhawa, DeAngelis, Freeman; 1990), (Fleet et al., 1994) Higher-order neural nets, Sigma-Pi-units Routing circuits (Olshausen; 1994) Bi-linear models (Tenenbaum, Freeman; 2000), (Grimes, Rao; 2005), (Olshausen; 2007) Subspace SOM (Kohonen, 1996) ISA, topographic ICA (Hyvarinen, Hoyer; 2000), (Karklin, Lewicki; 2003): Higher-order within image structure (2006 ) GBM, mcrbm, GAE, convisa, applications... Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
47 Multiplicative interactions Brief history of gating Mapping units (Hinton; 1981), dynamic mappings (v.d. Malsburg; 1981) Binocular+Motion Energy models (Adelson, Bergen; 1985), (Ozhawa, DeAngelis, Freeman; 1990), (Fleet et al., 1994) Higher-order neural nets, Sigma-Pi-units Routing circuits (Olshausen; 1994) Bi-linear models (Tenenbaum, Freeman; 2000), (Grimes, Rao; 2005), (Olshausen; 2007) Subspace SOM (Kohonen, 1996) ISA, topographic ICA (Hyvarinen, Hoyer; 2000), (Karklin, Lewicki; 2003): Higher-order within image structure (2006 ) GBM, mcrbm, GAE, convisa, applications... Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
48 Multiplicative interactions Brief history of gating Mapping units (Hinton; 1981), dynamic mappings (v.d. Malsburg; 1981) Binocular+Motion Energy models (Adelson, Bergen; 1985), (Ozhawa, DeAngelis, Freeman; 1990), (Fleet et al., 1994) Higher-order neural nets, Sigma-Pi-units Routing circuits (Olshausen; 1994) Bi-linear models (Tenenbaum, Freeman; 2000), (Grimes, Rao; 2005), (Olshausen; 2007) Subspace SOM (Kohonen, 1996) ISA, topographic ICA (Hyvarinen, Hoyer; 2000), (Karklin, Lewicki; 2003): Higher-order within image structure (2006 ) GBM, mcrbm, GAE, convisa, applications... Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
49 Multiplicative interactions Brief history of gating Mapping units (Hinton; 1981), dynamic mappings (v.d. Malsburg; 1981) Binocular+Motion Energy models (Adelson, Bergen; 1985), (Ozhawa, DeAngelis, Freeman; 1990), (Fleet et al., 1994) Higher-order neural nets, Sigma-Pi-units Routing circuits (Olshausen; 1994) Bi-linear models (Tenenbaum, Freeman; 2000), (Grimes, Rao; 2005), (Olshausen; 2007) Subspace SOM (Kohonen, 1996) ISA, topographic ICA (Hyvarinen, Hoyer; 2000), (Karklin, Lewicki; 2003): Higher-order within image structure (2006 ) GBM, mcrbm, GAE, convisa, applications... Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
50 Multiplicative interactions Brief history of gating Mapping units (Hinton; 1981), dynamic mappings (v.d. Malsburg; 1981) Binocular+Motion Energy models (Adelson, Bergen; 1985), (Ozhawa, DeAngelis, Freeman; 1990), (Fleet et al., 1994) Higher-order neural nets, Sigma-Pi-units Routing circuits (Olshausen; 1994) Bi-linear models (Tenenbaum, Freeman; 2000), (Grimes, Rao; 2005), (Olshausen; 2007) Subspace SOM (Kohonen, 1996) ISA, topographic ICA (Hyvarinen, Hoyer; 2000), (Karklin, Lewicki; 2003): Higher-order within image structure (2006 ) GBM, mcrbm, GAE, convisa, applications... Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
51 Multiplicative interactions Brief history of gating Mapping units (Hinton; 1981), dynamic mappings (v.d. Malsburg; 1981) Binocular+Motion Energy models (Adelson, Bergen; 1985), (Ozhawa, DeAngelis, Freeman; 1990), (Fleet et al., 1994) Higher-order neural nets, Sigma-Pi-units Routing circuits (Olshausen; 1994) Bi-linear models (Tenenbaum, Freeman; 2000), (Grimes, Rao; 2005), (Olshausen; 2007) Subspace SOM (Kohonen, 1996) ISA, topographic ICA (Hyvarinen, Hoyer; 2000), (Karklin, Lewicki; 2003): Higher-order within image structure (2006 ) GBM, mcrbm, GAE, convisa, applications... Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
52 Multiplicative interactions Brief history of gating Mapping units (Hinton; 1981), dynamic mappings (v.d. Malsburg; 1981) Binocular+Motion Energy models (Adelson, Bergen; 1985), (Ozhawa, DeAngelis, Freeman; 1990), (Fleet et al., 1994) Higher-order neural nets, Sigma-Pi-units Routing circuits (Olshausen; 1994) Bi-linear models (Tenenbaum, Freeman; 2000), (Grimes, Rao; 2005), (Olshausen; 2007) Subspace SOM (Kohonen, 1996) ISA, topographic ICA (Hyvarinen, Hoyer; 2000), (Karklin, Lewicki; 2003): Higher-order within image structure (2006 ) GBM, mcrbm, GAE, convisa, applications... Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
53 Mapping units 1981 (Hinton, 1981) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
54 Mapping units 1981 (Hinton, 1981) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
55 Example application: Action recognition (Hollywood 2) (Marszałek et al., 2009) Convolutional GBM (Taylor et al., 2010) hierarchical ISA (Le, et al., 2011) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
56 Mocap (Taylor, Hinton; 2009), (Taylor, et al.; 2010) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
57 Gated MRFs (Ranzato et al., 2010) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
58 Analogy making Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
59 Invariance aperture feature similarities image similarities Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
60 Outline 1 Introduction Feature Learning Correspondence in Computer Vision Multiview feature learning 2 Learning relational features Encoding relations Learning 3 Factorization, eigen-spaces and complex cells Factorization Eigen-spaces, energy models, complex cells 4 Applications and extensions Applications and extensions Conclusions Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
61 Outline 1 Introduction Feature Learning Correspondence in Computer Vision Multiview feature learning 2 Learning relational features Encoding relations Learning 3 Factorization, eigen-spaces and complex cells Factorization Eigen-spaces, energy models, complex cells 4 Applications and extensions Applications and extensions Conclusions Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
62 Sparse coding of images pairs? z? x y How to extend sparse coding to model relations? Sparse coding on the concatenation? Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
63 Sparse coding of images pairs? z? x y How to extend sparse coding to model relations? Sparse coding on the concatenation? Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
64 Sparse coding on the concatenation? A case study: Translations of binary, one-d images. Suppose images are random and can change in one of three ways: Example Image x: Possible Image y: Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
65 Sparse coding on the concatenation? A hidden variable that collects evidence for a shift to the right. What if the images are random or noisy? Can we pool over more than one pixel? z k Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
66 Sparse coding on the concatenation? A hidden variable that collects evidence for a shift to the right. What if the images are random or noisy? Can we pool over more than one pixel? z k Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
67 Sparse coding on the concatenation? A hidden variable that collects evidence for a shift to the right. What if the images are random or noisy? Can we pool over more than one pixel? z k? Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
68 Sparse coding on the concatenation? Obviously not, because now the hidden unit would get equally happy if it would see the non-shift (second pixel from the left). The problem: Hidden variables act like OR-gates, that accumulate evidence. z k? Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
69 Cross-products Intuitively, what we need instead are logical ANDs, which can represent coincidences (eg. Zetzsche et al., 2003, 2005). This amounts to using the outer product L := outer(x, y): We can unroll this matrix, and let this be the data: z k w ijk Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
70 Cross-products Each component L ij of the outer-product matrix will constitute evidence for exactly one type of shift. Hiddens pool over products of pixels. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
71 Cross-products Each component L ij of the outer-product matrix will constitute evidence for exactly one type of shift. Hiddens pool over products of pixels. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
72 Cross-products Each component L ij of the outer-product matrix will constitute evidence for exactly one type of shift. Hiddens pool over products of pixels. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
73 A different view: Families of manifolds y x Feature learning reveals the (local) manifold structure in data. When y is a transformed version of x, we can still think of y as being confined to a manifold, but it will be a conditional manifold. Idea: Learn a model for y, but let parameters be a function of x. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
74 Conditional inference z z k x i y j x y Inferring z If we use a linear function, w jk (x) = i w ijkx i, we get z k = w jk y j = ( ) w ijk x i yj = w ijk x i y j j j i ij Inference via bilinear function of the inputs. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
75 Conditional inference z z k x i y j x y Inferring y To infer y: y j = k w jk z k = k ( ) w ijk x i zk = i ik w ijk x i z k Inference via bilinear function of x, z. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
76 Input-modulated filters z z k x i y j x y This is feature learning with input-dependent weights. Input pixels can vote for features in the output image. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
77 A different visualization z z k x i y j x y A hidden can blend in one slice W k of the parameter tensor. A slice does linear regression in pixel space. So for binary hiddens, this is a mixture of 2 K image warps. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
78 Outline 1 Introduction Feature Learning Correspondence in Computer Vision Multiview feature learning 2 Learning relational features Encoding relations Learning 3 Factorization, eigen-spaces and complex cells Factorization Eigen-spaces, energy models, complex cells 4 Applications and extensions Applications and extensions Conclusions Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
79 Learning is predictive coding z z k x i y j Predictive sparse coding The cost for a training pair (x, y) is: x ( yj w ijk x i z k ) 2 j Training as usual: Infer z, update W. (Tenenbaum, Freeman; 2000), (Grimes, Rao; 2005), (Olshausen; 2007), (Memisevic, Hinton; 2007) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163 ik y
80 Example: Gated Boltzmann machine z z k x i y j x y Three-way RBM (Memisevic, Hinton; 2007) E(x, y, z) = ijk w ijk x i y j z k p(y, z x) = 1 Z(x) exp ( E(x, y, z) ), Z(x) = y,z exp ( E(x, y, z) ) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
81 Example: Gated Boltzmann machine z z k x i y j x y Three-way RBM (Memisevic, Hinton; 2007) E(x, y, z) = ijk w ijk x i y j z k p(y, z x) = 1 Z(x) exp ( E(x, y, z) ), Z(x) = y,z exp ( E(x, y, z) ) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
82 Example: Gated Boltzmann machine z z k x i y j x y Three-way RBM (Memisevic, Hinton; 2007) p(z k x, y) = sigmoid( ij p(y j x, z) = sigmoid( ik W ijk x i y j ) W ijk x i z k ) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
83 Example: Gated auto-encoder ŷ j ŷ x x i z k z y j y Gated autoencoders Turn encoder and decoder weights into functions of x. Learning the same as in a standard auto-encoder for y. The model is still a DAG, so back-prop works exactly like in a standard autoencoder. (Memisevic, 2011) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
84 Toy example: Conditionally trained Hidden flow-fields Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
85 Toy example: Conditionally trained Hidden flow-fields, inhibitory connections Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
86 Toy example: Learning optical flow x y z x test y(x test, z) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
87 Combinatorial flowfields x y z x test y(x test, z) Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
88 Joint training z zk xi yj x y Conditional training makes it hard to answer questions like: How likely are the given images transforms of one another? To answer questions like these, we require a joint image model, p(x, y z), given mapping units. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
89 Joint training z z k E(x, y, z) = ijk w ijk x i y j z k x i y j p(x, y, z) = 1 Z exp ( E(x, y, z) ) Z = exp ( E(x, y, z) ) x,y,z x y Use three-way sampling in a Gated Boltzmann Machine (Susskind et al., 2011). Can apply this to matching tasks (more later). Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
90 Joint training z z k E(x, y, z) = ijk w ijk x i y j z k x i y j p(x, y, z) = 1 Z exp ( E(x, y, z) ) Z = exp ( E(x, y, z) ) x,y,z x y Use three-way sampling in a Gated Boltzmann Machine (Susskind et al., 2011). Can apply this to matching tasks (more later). Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
91 Joint training ŷ j ŷ x x i z k z y j y For the autoencoder we can use a simple hack: Add up two conditional costs: ( j yj ik w ijkx i z k ) 2 + ( i xi jk w ijky j z k ) 2 Force parameters to transform in both directions. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
92 Joint training ˆx i ˆx y y j z k z x i x For the autoencoder we can use a simple hack: Add up two conditional costs: ( j yj ik w ijkx i z k ) 2 + ( i xi jk w ijky j z k ) 2 Force parameters to transform in both directions. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
93 Pool over products Take-home message To gather evidence for a transformation, let hidden units compute the sum over products of input components. Roland Memisevic (Frankfurt, Montreal) Multiview Feature Learning Tutorial at IPAM / 163
Deep Learning in Image Processing
Deep Learning in Image Processing Roland Memisevic University of Montreal & TwentyBN ICISP 2016 Roland Memisevic Deep Learning in Image Processing ICISP 2016 f 2? cathedral high-rise f 1 It s the features,
On spatio-temporal sparse coding: Analysis and an algorithm
On spatio-temporal sparse coding: Analysis and an algorithm Roland Memisevic Department of Computer Science University of Frankfurt ro@cs.uni-frankfurt.de Abstract Sparse coding is a common approach to
Learning visual odometry with a convolutional network
Learning visual odometry with a convolutional network Kishore Konda 1, Roland Memisevic 2 1 Goethe University Frankfurt 2 University of Montreal konda.kishorereddy@gmail.com, roland.memisevic@gmail.com
Learning-based Methods in Vision
Learning-based Methods in Vision 16-824 Sparsity and Deep Learning Motivation Multitude of hand-designed features currently in use in vision - SIFT, HoG, LBP, MSER, etc. Even the best approaches, just
Learning invariant features by harnessing the aperture problem
Roland Memisevic University of Montreal Georgios Exarchakis University of California, Berkeley memisevr@iro.umontreal.ca exarchakis@berkeley.edu Abstract The energy model is a simple, biologically inspired
Learning Class-Specific Image Transformations with Higher-Order Boltzmann Machines
Learning Class-Specific Image Transformations with Higher-Order Boltzmann Machines Gary B. Huang Erik Learned-Miller University of Massachusetts Amherst Amherst, MA {gbhuang,elm}@cs.umass.edu Abstract
On multi-view feature learning
Roland Memisevic University of Frankfurt, Robert-Mayer-Str. 10, 60325 Frankfurt, Germany ro@cs.uni-frankfurt.de Abstract Sparse coding is a common approach to learning local features for object recognition.
Learning to Represent Spatial Transformations with Factored Higher-Order Boltzmann Machines
LETTER Communicated by Dana Ballard Learning to Represent Spatial Transformations with Factored Higher-Order Boltzmann Machines Roland Memisevic roland@cs.toronto.edu Geoffrey E. Hinton hinton@cs.toronto.edu
Implicit Mixtures of Restricted Boltzmann Machines
Implicit Mixtures of Restricted Boltzmann Machines Vinod Nair and Geoffrey Hinton Department of Computer Science, University of Toronto 10 King s College Road, Toronto, M5S 3G5 Canada {vnair,hinton}@cs.toronto.edu
Day 3 Lecture 1. Unsupervised Learning
Day 3 Lecture 1 Unsupervised Learning Semi-supervised and transfer learning Myth: you can t do deep learning unless you have a million labelled examples for your problem. Reality You can learn useful representations
Deep Learning. Volker Tresp Summer 2014
Deep Learning Volker Tresp Summer 2014 1 Neural Network Winter and Revival While Machine Learning was flourishing, there was a Neural Network winter (late 1990 s until late 2000 s) Around 2010 there
Unsupervised Learning of Image Transformations
Department of Computer Science 6 King s College Rd, Toronto University of Toronto M5S 3G4, Canada http://learning.cs.toronto.edu fax: +1 416 978 1455 Copyright c Roland Memisevic & Geoffrey Hinton 2006
Deep Generative Models Variational Autoencoders
Deep Generative Models Variational Autoencoders Sudeshna Sarkar 5 April 2017 Generative Nets Generative models that represent probability distributions over multiple variables in some way. Directed Generative
Energy Based Models, Restricted Boltzmann Machines and Deep Networks. Jesse Eickholt
Energy Based Models, Restricted Boltzmann Machines and Deep Networks Jesse Eickholt ???? Who s heard of Energy Based Models (EBMs) Restricted Boltzmann Machines (RBMs) Deep Belief Networks Auto-encoders
COMP 551 Applied Machine Learning Lecture 16: Deep Learning
COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all
CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]
![CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]](/thumbs/89/100746783.jpg "CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]")
CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, 2015. 11:59pm, PDF to Canvas [100 points] Instructions. Please write up your responses to the following problems clearly and concisely.
Does the Brain do Inverse Graphics?
Does the Brain do Inverse Graphics? Geoffrey Hinton, Alex Krizhevsky, Navdeep Jaitly, Tijmen Tieleman & Yichuan Tang Department of Computer Science University of Toronto How to learn many layers of features
Deep Learning. Deep Learning. Practical Application Automatically Adding Sounds To Silent Movies
http://blog.csdn.net/zouxy09/article/details/8775360 Automatic Colorization of Black and White Images Automatically Adding Sounds To Silent Movies Traditionally this was done by hand with human effort
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images Marc Aurelio Ranzato Yann LeCun Courant Institute of Mathematical Sciences New York University - New York, NY 10003 Abstract
Deep Learning for Generic Object Recognition
Deep Learning for Generic Object Recognition, Computational and Biological Learning Lab The Courant Institute of Mathematical Sciences New York University Collaborators: Marc'Aurelio Ranzato, Fu Jie Huang,
Human Vision Based Object Recognition Sye-Min Christina Chan
Human Vision Based Object Recognition Sye-Min Christina Chan Abstract Serre, Wolf, and Poggio introduced an object recognition algorithm that simulates image processing in visual cortex and claimed to
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images Marc Aurelio Ranzato Yann LeCun Courant Institute of Mathematical Sciences New York University - New York, NY 10003 Abstract
Learning Feature Hierarchies for Object Recognition
Learning Feature Hierarchies for Object Recognition Koray Kavukcuoglu Computer Science Department Courant Institute of Mathematical Sciences New York University Marc Aurelio Ranzato, Kevin Jarrett, Pierre
Unsupervised Learning of Feature Hierarchies
Unsupervised Learning of Feature Hierarchies by Marc Aurelio Ranzato A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Computer Science
Modeling image patches with a directed hierarchy of Markov random fields
Modeling image patches with a directed hierarchy of Markov random fields Simon Osindero and Geoffrey Hinton Department of Computer Science, University of Toronto 6, King s College Road, M5S 3G4, Canada
Gated Boltzmann Machine in Texture Modeling
Gated Boltzmann Machine in Texture Modeling Tele Hao, Tapani Rao, Alexander Ilin, and Juha Karhunen Department of Information and Computer Science Aalto University, Espoo, Finland firstname.lastname@aalto.fi
Self-Organizing Sparse Codes
Self-Organizing Sparse Codes Yangqing Jia Sergey Karayev 2010-12-16 Abstract Sparse coding as applied to natural image patches learns Gabor-like components that resemble those found in the lower areas
Neural Networks for unsupervised learning From Principal Components Analysis to Autoencoders to semantic hashing
Neural Networks for unsupervised learning From Principal Components Analysis to Autoencoders to semantic hashing feature 3 PC 3 Beate Sick Many slides are taken form Hinton s great lecture on NN: https://www.coursera.org/course/neuralnets
C. Poultney S. Cho pra (NYU Courant Institute) Y. LeCun
Efficient Learning of Sparse Overcomplete Representations with an Energy-Based Model Marc'Aurelio Ranzato C. Poultney S. Cho pra (NYU Courant Institute) Y. LeCun CIAR Summer School Toronto 2006 Why Extracting
Multiple-Choice Questionnaire Group C
Family name: Vision and Machine-Learning Given name: 1/28/2011 Multiple-Choice naire Group C No documents authorized. There can be several right answers to a question. Marking-scheme: 2 points if all right
Does the Brain do Inverse Graphics?
Does the Brain do Inverse Graphics? Geoffrey Hinton, Alex Krizhevsky, Navdeep Jaitly, Tijmen Tieleman & Yichuan Tang Department of Computer Science University of Toronto The representation used by the
An Analysis of Single-Layer Networks in Unsupervised Feature Learning
An Analysis of Single-Layer Networks in Unsupervised Feature Learning Adam Coates Honglak Lee Andrew Y. Ng Stanford University Computer Science Dept. 353 Serra Mall Stanford, CA 94305 University of Michigan
Neural Network and Deep Learning. Donglin Zeng, Department of Biostatistics, University of North Carolina
Neural Network and Deep Learning Early history of deep learning Deep learning dates back to 1940s: known as cybernetics in the 1940s-60s, connectionism in the 1980s-90s, and under the current name starting
Machine Learning. Deep Learning. Eric Xing (and Pengtao Xie) , Fall Lecture 8, October 6, Eric CMU,
Machine Learning 10-701, Fall 2015 Deep Learning Eric Xing (and Pengtao Xie) Lecture 8, October 6, 2015 Eric Xing @ CMU, 2015 1 A perennial challenge in computer vision: feature engineering SIFT Spin image
Learning global properties of scene images based on their correlational structures
Learning global properties of scene images based on their correlational structures Wooyoung Lee Machine Learning Department Carnegie Mellon University Pittsburgh, PA 15213 wooyoung@cs.cmu.edu Michael S.
Stacked Denoising Autoencoders for Face Pose Normalization
Stacked Denoising Autoencoders for Face Pose Normalization Yoonseop Kang 1, Kang-Tae Lee 2,JihyunEun 2, Sung Eun Park 2 and Seungjin Choi 1 1 Department of Computer Science and Engineering Pohang University
Multi-view Facial Expression Recognition Analysis with Generic Sparse Coding Feature
0/19.. Multi-view Facial Expression Recognition Analysis with Generic Sparse Coding Feature Usman Tariq, Jianchao Yang, Thomas S. Huang Department of Electrical and Computer Engineering Beckman Institute
Classification of objects from Video Data (Group 30)
Classification of objects from Video Data (Group 30) Sheallika Singh 12665 Vibhuti Mahajan 12792 Aahitagni Mukherjee 12001 M Arvind 12385 1 Motivation Video surveillance has been employed for a long time
Learning Spatio-temporally Invariant Representations from Video
Learning Spatio-temporally Invariant Representations from Video Jae Hyun Lim, Hansol Choi, Jun-Cheol Park, Jae Young Jun and Dae-shik Kim Department of Electrical Engineering Korea Advanced Institute of
Efficient Feature Learning Using Perturb-and-MAP
Efficient Feature Learning Using Perturb-and-MAP Ke Li, Kevin Swersky, Richard Zemel Dept. of Computer Science, University of Toronto {keli,kswersky,zemel}@cs.toronto.edu Abstract Perturb-and-MAP [1] is
3D Object Recognition with Deep Belief Nets
3D Object Recognition with Deep Belief Nets Vinod Nair and Geoffrey E. Hinton Department of Computer Science, University of Toronto 10 King s College Road, Toronto, M5S 3G5 Canada {vnair,hinton}@cs.toronto.edu
Artificial Neural Networks. Introduction to Computational Neuroscience Ardi Tampuu
Artificial Neural Networks Introduction to Computational Neuroscience Ardi Tampuu 7.0.206 Artificial neural network NB! Inspired by biology, not based on biology! Applications Automatic speech recognition
Advanced Introduction to Machine Learning, CMU-10715
Advanced Introduction to Machine Learning, CMU-10715 Deep Learning Barnabás Póczos, Sept 17 Credits Many of the pictures, results, and other materials are taken from: Ruslan Salakhutdinov Joshua Bengio
Facial Expression Classification with Random Filters Feature Extraction
Facial Expression Classification with Random Filters Feature Extraction Mengye Ren Facial Monkey mren@cs.toronto.edu Zhi Hao Luo It s Me lzh@cs.toronto.edu I. ABSTRACT In our work, we attempted to tackle
LEARNING A SPARSE DICTIONARY OF VIDEO STRUCTURE FOR ACTIVITY MODELING. Nandita M. Nayak, Amit K. Roy-Chowdhury. University of California, Riverside
LEARNING A SPARSE DICTIONARY OF VIDEO STRUCTURE FOR ACTIVITY MODELING Nandita M. Nayak, Amit K. Roy-Chowdhury University of California, Riverside ABSTRACT We present an approach which incorporates spatiotemporal
Tiled convolutional neural networks
Tiled convolutional neural networks Quoc V. Le, Jiquan Ngiam, Zhenghao Chen, Daniel Chia, Pang Wei Koh, Andrew Y. Ng Computer Science Department, Stanford University {quocle,jngiam,zhenghao,danchia,pangwei,ang}@cs.stanford.edu
Semi-Supervised Hierarchical Models for 3D Human Pose Reconstruction
Semi-Supervised Hierarchical Models for 3D Human Pose Reconstruction Atul Kanaujia, CBIM, Rutgers Cristian Sminchisescu, TTI-C Dimitris Metaxas,CBIM, Rutgers 3D Human Pose Inference Difficulties Towards
Why equivariance is better than premature invariance
1 Why equivariance is better than premature invariance Geoffrey Hinton Canadian Institute for Advanced Research & Department of Computer Science University of Toronto with contributions from Sida Wang
Graphical Models for Computer Vision
Graphical Models for Computer Vision Pedro F Felzenszwalb Brown University Joint work with Dan Huttenlocher, Joshua Schwartz, Ross Girshick, David McAllester, Deva Ramanan, Allie Shapiro, John Oberlin
Modeling and Monitoring Crop Disease in Developing Countries
Modeling and Monitoring Crop Disease in Developing Countries John Quinn 1, Kevin Leyton-Brown 2, Ernest Mwebaze 1 1 Department of Computer Science 2 Department of Computer Science Makerere University,
Lecture 21 : A Hybrid: Deep Learning and Graphical Models
10-708: Probabilistic Graphical Models, Spring 2018 Lecture 21 : A Hybrid: Deep Learning and Graphical Models Lecturer: Kayhan Batmanghelich Scribes: Paul Liang, Anirudha Rayasam 1 Introduction and Motivation
Computer vision: models, learning and inference. Chapter 13 Image preprocessing and feature extraction
Computer vision: models, learning and inference Chapter 13 Image preprocessing and feature extraction Preprocessing The goal of pre-processing is to try to reduce unwanted variation in image due to lighting,
Deep Learning. Volker Tresp Summer 2015
Deep Learning Volker Tresp Summer 2015 1 Neural Network Winter and Revival While Machine Learning was flourishing, there was a Neural Network winter (late 1990 s until late 2000 s) Around 2010 there
Neural Networks for Machine Learning. Lecture 15a From Principal Components Analysis to Autoencoders
Neural Networks for Machine Learning Lecture 15a From Principal Components Analysis to Autoencoders Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed Principal Components
3 : Representation of Undirected GMs
0-708: Probabilistic Graphical Models 0-708, Spring 202 3 : Representation of Undirected GMs Lecturer: Eric P. Xing Scribes: Nicole Rafidi, Kirstin Early Last Time In the last lecture, we discussed directed
Bilinear Models of Natural Images
In: SPIE Proceedings vol. 6492: Human Vision and Electronic Imaging XII, (B.E. Rogowitz, T.N. Pappas, S.J. Daly, Eds.), Jan 28-Feb 1, 2007, San Jose, California Bilinear Models of Natural Images Bruno
Dynamic Routing Between Capsules
Report Explainable Machine Learning Dynamic Routing Between Capsules Author: Michael Dorkenwald Supervisor: Dr. Ullrich Köthe 28. Juni 2018 Inhaltsverzeichnis 1 Introduction 2 2 Motivation 2 3 CapusleNet
Learning Transformational Invariants from Natural Movies
Learning Transformational Invariants from Natural Movies Charles F. Cadieu & Bruno A. Olshausen Helen Wills Neuroscience Institute University of California, Berkeley Berkeley, CA 94720 {cadieu, baolshausen}@berkeley.edu
Deep (1) Matthieu Cord LIP6 / UPMC Paris 6
Deep (1) Matthieu Cord LIP6 / UPMC Paris 6 Syllabus 1. Whole traditional (old) visual recognition pipeline 2. Introduction to Neural Nets 3. Deep Nets for image classification To do : Voir la leçon inaugurale
Deep Learning. Deep Learning provided breakthrough results in speech recognition and image classification. Why?
Data Mining Deep Learning Deep Learning provided breakthrough results in speech recognition and image classification. Why? Because Speech recognition and image classification are two basic examples of
CPSC340. State-of-the-art Neural Networks. Nando de Freitas November, 2012 University of British Columbia
CPSC340 State-of-the-art Neural Networks Nando de Freitas November, 2012 University of British Columbia Outline of the lecture This lecture provides an overview of two state-of-the-art neural networks:
Feature Descriptors. CS 510 Lecture #21 April 29 th, 2013
Feature Descriptors CS 510 Lecture #21 April 29 th, 2013 Programming Assignment #4 Due two weeks from today Any questions? How is it going? Where are we? We have two umbrella schemes for object recognition
Dropout. Sargur N. Srihari This is part of lecture slides on Deep Learning:
Dropout Sargur N. srihari@buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Regularization Strategies 1. Parameter Norm Penalties 2. Norm Penalties
Algorithms for Markov Random Fields in Computer Vision
Algorithms for Markov Random Fields in Computer Vision Dan Huttenlocher November, 2003 (Joint work with Pedro Felzenszwalb) Random Field Broadly applicable stochastic model Collection of n sites S Hidden
Deep Convolutional Neural Networks. Nov. 20th, 2015 Bruce Draper
Deep Convolutional Neural Networks Nov. 20th, 2015 Bruce Draper Background: Fully-connected single layer neural networks Feed-forward classification Trained through back-propagation Example Computer Vision
Bilevel Sparse Coding
Adobe Research 345 Park Ave, San Jose, CA Mar 15, 2013 Outline 1 2 The learning model The learning algorithm 3 4 Sparse Modeling Many types of sensory data, e.g., images and audio, are in high-dimensional
The Lucas & Kanade Algorithm
The Lucas & Kanade Algorithm Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Registration, Registration, Registration. Linearizing Registration. Lucas & Kanade Algorithm. 3 Biggest
Deep Learning Shape Priors for Object Segmentation
2013 IEEE Conference on Computer Vision and Pattern Recognition Deep Learning Shape Priors for Object Segmentation Fei Chen a,c Huimin Yu a,b Roland Hu a Xunxun Zeng d a Department of Information Science
The "tree-dependent components" of natural scenes are edge filters
The "tree-dependent components" of natural scenes are edge filters Daniel Zoran Interdisciplinary Center for Neural Computation Hebrew University of Jerusalem daniez@cs.huji.ac.il Yair Weiss School of
08 An Introduction to Dense Continuous Robotic Mapping
NAVARCH/EECS 568, ROB 530 - Winter 2018 08 An Introduction to Dense Continuous Robotic Mapping Maani Ghaffari March 14, 2018 Previously: Occupancy Grid Maps Pose SLAM graph and its associated dense occupancy
Deep Learning Applications
October 20, 2017 Overview Supervised Learning Feedforward neural network Convolution neural network Recurrent neural network Recursive neural network (Recursive neural tensor network) Unsupervised Learning
Adversarially Learned Inference
Institut des algorithmes d apprentissage de Montréal Adversarially Learned Inference Aaron Courville CIFAR Fellow Université de Montréal Joint work with: Vincent Dumoulin, Ishmael Belghazi, Olivier Mastropietro,
Restricted Boltzmann Machines. Shallow vs. deep networks. Stacked RBMs. Boltzmann Machine learning: Unsupervised version
Shallow vs. deep networks Restricted Boltzmann Machines Shallow: one hidden layer Features can be learned more-or-less independently Arbitrary function approximator (with enough hidden units) Deep: two
Partial Least Squares Regression on Grassmannian Manifold for Emotion Recognition
Emotion Recognition In The Wild Challenge and Workshop (EmotiW 2013) Partial Least Squares Regression on Grassmannian Manifold for Emotion Recognition Mengyi Liu, Ruiping Wang, Zhiwu Huang, Shiguang Shan,
Capsule Networks. Eric Mintun
Capsule Networks Eric Mintun Motivation An improvement* to regular Convolutional Neural Networks. Two goals: Replace max-pooling operation with something more intuitive. Keep more info about an activated
Batch Normalization Accelerating Deep Network Training by Reducing Internal Covariate Shift. Amit Patel Sambit Pradhan
Batch Normalization Accelerating Deep Network Training by Reducing Internal Covariate Shift Amit Patel Sambit Pradhan Introduction Internal Covariate Shift Batch Normalization Computational Graph of Batch
Extracting and Composing Robust Features with Denoising Autoencoders
Presenter: Alexander Truong March 16, 2017 Extracting and Composing Robust Features with Denoising Autoencoders Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre-Antoine Manzagol 1 Outline Introduction
Deep Learning in Visual Recognition. Thanks Da Zhang for the slides
Deep Learning in Visual Recognition Thanks Da Zhang for the slides Deep Learning is Everywhere 2 Roadmap Introduction Convolutional Neural Network Application Image Classification Object Detection Object
A Dendrogram. Bioinformatics (Lec 17)
A Dendrogram 3/15/05 1 Hierarchical Clustering [Johnson, SC, 1967] Given n points in R d, compute the distance between every pair of points While (not done) Pick closest pair of points s i and s j and
Cambridge Interview Technical Talk
Cambridge Interview Technical Talk February 2, 2010 Table of contents Causal Learning 1 Causal Learning Conclusion 2 3 Motivation Recursive Segmentation Learning Causal Learning Conclusion Causal learning
Unsupervised Discovery of Non-Linear Structure using Contrastive Backpropagation
Unsupervised Discovery of Non-Linear Structure using Contrastive Backpropagation G. E. Hinton, S. Osindero, M. Welling and Y. W. Teh Department of Computer Science University of Toronto Toronto, Canada
Learning Visual Semantics: Models, Massive Computation, and Innovative Applications
Learning Visual Semantics: Models, Massive Computation, and Innovative Applications Part II: Visual Features and Representations Liangliang Cao, IBM Watson Research Center Evolvement of Visual Features
Introduction to Deep Learning
ENEE698A : Machine Learning Seminar Introduction to Deep Learning Raviteja Vemulapalli Image credit: [LeCun 1998] Resources Unsupervised feature learning and deep learning (UFLDL) tutorial (http://ufldl.stanford.edu/wiki/index.php/ufldl_tutorial)
Final Exam Study Guide
Final Exam Study Guide Exam Window: 28th April, 12:00am EST to 30th April, 11:59pm EST Description As indicated in class the goal of the exam is to encourage you to review the material from the course.
Recurrent Convolutional Neural Networks for Scene Labeling
Recurrent Convolutional Neural Networks for Scene Labeling Pedro O. Pinheiro, Ronan Collobert Reviewed by Yizhe Zhang August 14, 2015 Scene labeling task Scene labeling: assign a class label to each pixel
Sparsity priors and boosting for learning localized distributed feature representations
Technical Report TR-2010-04 University of British Columbia Sparsity priors and boosting for learning localized distributed feature representations Bo Chen Kevin Swersky Ben Marlin Nando de Freitas March
Detecting Salient Contours Using Orientation Energy Distribution. Part I: Thresholding Based on. Response Distribution
Detecting Salient Contours Using Orientation Energy Distribution The Problem: How Does the Visual System Detect Salient Contours? CPSC 636 Slide12, Spring 212 Yoonsuck Choe Co-work with S. Sarma and H.-C.
ImageCLEF 2011
SZTAKI @ ImageCLEF 2011 Bálint Daróczy joint work with András Benczúr, Róbert Pethes Data Mining and Web Search Group Computer and Automation Research Institute Hungarian Academy of Sciences Training/test
Modeling pigeon behaviour using a Conditional Restricted Boltzmann Machine
Modeling pigeon behaviour using a Conditional Restricted Boltzmann Machine Matthew D. Zeiler 1,GrahamW.Taylor 1, Nikolaus F. Troje 2 and Geoffrey E. Hinton 1 1- University of Toronto - Dept. of Computer
Large-scale Video Classification with Convolutional Neural Networks
Large-scale Video Classification with Convolutional Neural Networks Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, Li Fei-Fei Note: Slide content mostly from : Bay Area
A SPARSE ENCODING SYMMETRIC MACHINES PRE-TRAINING FOR TEMPORAL DEEP BELIEF NETWORKS FOR MOTION ANALYSIS AND SYNTHESIS
A SPARSE ENCODING SYMMETRIC MACHINES PRE-TRAINING FOR TEMPORAL DEEP BELIEF NETWORKS FOR MOTION ANALYSIS AND SYNTHESIS 1 MILYUN NI MA SHOUMI, MOHAMAD IVAN FANANY Faculty of Computer Science, University
ECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
LSTM and its variants for visual recognition. Xiaodan Liang Sun Yat-sen University
LSTM and its variants for visual recognition Xiaodan Liang xdliang328@gmail.com Sun Yat-sen University Outline Context Modelling with CNN LSTM and its Variants LSTM Architecture Variants Application in
Neural Network Neurons
Neural Networks Neural Network Neurons 1 Receives n inputs (plus a bias term) Multiplies each input by its weight Applies activation function to the sum of results Outputs result Activation Functions Given
Sparse Models in Image Understanding And Computer Vision
Sparse Models in Image Understanding And Computer Vision Jayaraman J. Thiagarajan Arizona State University Collaborators Prof. Andreas Spanias Karthikeyan Natesan Ramamurthy Sparsity Sparsity of a vector
Markov Networks in Computer Vision
Markov Networks in Computer Vision Sargur Srihari srihari@cedar.buffalo.edu 1 Markov Networks for Computer Vision Some applications: 1. Image segmentation 2. Removal of blur/noise 3. Stereo reconstruction
Handwritten Hindi Numerals Recognition System
CS365 Project Report Handwritten Hindi Numerals Recognition System Submitted by: Akarshan Sarkar Kritika Singh Project Mentor: Prof. Amitabha Mukerjee 1 Abstract In this project, we consider the problem
IMAGE RETRIEVAL USING VLAD WITH MULTIPLE FEATURES
IMAGE RETRIEVAL USING VLAD WITH MULTIPLE FEATURES Pin-Syuan Huang, Jing-Yi Tsai, Yu-Fang Wang, and Chun-Yi Tsai Department of Computer Science and Information Engineering, National Taitung University,
Learning Tractable Probabilistic Models Pedro Domingos
Learning Tractable Probabilistic Models Pedro Domingos Dept. Computer Science & Eng. University of Washington 1 Outline Motivation Probabilistic models Standard tractable models The sum-product theorem
Research on Pruning Convolutional Neural Network, Autoencoder and Capsule Network
Research on Pruning Convolutional Neural Network, Autoencoder and Capsule Network Tianyu Wang Australia National University, Colledge of Engineering and Computer Science u@anu.edu.au Abstract. Some tasks,