The Sparse Manifold Transform

Transcription

1 The Sparse Manifold Transform Bruno Olshausen Redwood Center for Theoretical Neuroscience, Helen Wills Neuroscience Institute, and School of Optometry UC Berkeley with Yubei Chen (EECS) and Dylan Paiton (Vision Science) NSF RI: Extracting and understanding sparse structure in spatiotemporal data, PI: Sommer

2 Dylan Paiton Yubei Chen Redwood Center for Theoretical Neuroscience - April 2016

3 Are there principles? God is a hacker Francis Crick Individual nerve cells were formerly thought to be unreliable This was quite wrong, and we now realise their apparently erratic behavior was caused by our ignorance, not the neuron s incompetence. H.B. Barlow (1972)

4 Wallisch & Movshon (2008) Gabor filters..?.. objects. faces

5 Three principles of unsupervised learning Sparse coding feature selectivity Manifold flattening equivariance Persistence invariance

6 Sparse coding

7 The Ratio Club (1952) Horace Barlow

8

9 V1 is highly overcomplete LGN afferents IVb layer 4 cortex Barlow (1981) 0 1mm C I b and IV

10 ... Sparse coding External world Internal model image model (Olshausen & Field, 1996; Chen, Donoho & Saunders 1995) I(x,y) φ i (x,y) ai MX I( x) = a i i ( x)+ ( x) i=1 image neural features other activities stuff (sparse)

11 Energy function preserve information be sparse

12 <latexit sha1_base64="4yfjz/utdtr7dp3o4qvddyy2i2i=">aaab6xicbvbns8naej3ur1q/qh69lbbbu0le0gpri8cq9gpaudbbsbt0swm7g6ge/gmvhhtx6j/y5r9x0+agrq8ghu/nmdmvsatxxnw/ndla+sbmvnm7sro7t39qptxq6zhvdfssfrhqblsj4bjbhhub3uqhjqkbnwbym/udj1sax/lrtbp0izqsposmgis9jcqdas2tu3oqveivpayfmopqv38yszrcazigwvc8nzf+rpxhtocs0k81jprn6ah7lkoaofaz+auzcmaviqljzusamld/t2q00noabbyzomasl71c/m/rpsa89jmuk9sgzitfysqiiun+nhlyhcyiqswukw5vjwxmfwxghpoh4c2/veraf3xprxv3l7xgtrfhgu7gfm7bgytowb00oqumqnigv3hzjs6l8+58lfpltjfzdh/gfp4aapem/w==</latexit> Locally Competitive Algorithm (LCA) minimizes the energy function (Rozell, Johnson, Baraniuk & Olshausen, 2008) b i = X x i(x) I(x) G ij = X x i(x) j (x)

13 Learned dictionary { i } spatial frequency (cycles/window) y x Olshausen & Field (1996)

14 Sparse encoding of a time-varying image image sparse encoding reconstruction

15 Sparse coefficient activations form smooth trajectories in particular local subspaces

16 Basis functions tile the manifold of natural images in such a way that data points along the manifold are spanned by a small number of basis vectors.

17 Manifold flattening

18 Science, 22 Dec. 2000

19 Local Linear Embedding (LLE) exactly such transformations. We th expect their characterization of local g try in the original data space to be valid for local patches on the manif particular, the same weights W ij that struct the ith data point in D dime should also reconstruct its embedded fold coordinates in d dimensions. LLE constructs a neighborhood-pre mapping based on the above idea. In th step of the algorithm, each high-dime ε W X i j W ijxj 2 observation X i is mapped to a low-dime vector Yi representing i global internal nates on the manifold. This is done by ch d-dimensional coordinates Y i to minim embedding cost function Y i Y i j W ij Yj 2 This cost function, like the previous based on locally linear reconstruction but here we fix the weights W whil

20 <latexit sha1_base64="nco/reiwdp/2joqbhwmf/bk9w30=">aaab6nicbvbns8naej3ur1q/oh69lbbbu0le0gpri8ek9gpaudbbtbt0swm7e6ge/gqvhhtx6i/y5r9x2+agrq8ghu/nmdmvtkuw6hnftmltfwnzq7xd2dnd2z9wd49ajsk0402wyer3qmq4fio3uadknvrzgoest8px7cxvp3ftrkieczlyikzdjslbkfrpodexfbfq1bw5ycrxc1kfao2++9ubjcyluuimqtfd30sxyklgwssfvnqz4sllyzrkxusvjbkj8vmpu3jmlqgjem1lizmrvydyghsziupbgvmcmwvvjv7ndtomroncqdrdrthiuzrjggmz/u0gqnogcmijzvrywwkbuu0z2nqqngr/+evv0rqo+v7nv7+s1m+kompwaqdwdj5cqr3uoafnydcez3ifn0c6l86787foltnfzdh8gfp5ayogjba=</latexit> Local Linear Landmarks (LLL) (Vladymyrov & Carreira-Perpinán, 2013)

21 From 1-sparse to k-sparse 1-sparse k-sparse Interpolation by KNN data vectors Interpolation by landmarks or dictionary

22 Topographic ICA (Hyvärinen & Hoyer 2001)

23 f filters as a function of time (horizontal axis) a Hyvärinen et al. Vol. 20, No. 7/July 2003/J. Opt. Soc. Am. A Bubbles: a unifying framework for low-level statistical properties of natural image sequences Aapo Hyvärinen, Jarmo Hurri, and Jaakko Väyrynen Neural Networks Research Centre, Helsinki University of Technology, P.O. Box 9800, FIN HUT, Finland

24 Sparse Manifold Transform (Yubei Chen, Ph.D. thesis) The Sparse Manifold Transform x (R2 )... 1 <latexit sha1_base64="5gzcxzr7alhkdfo5i9imea1bwm8=">aaacghicbvdlsgmxfm3uv62vqks3wsiiap0pgoiirtcuk9ghnlxcsdm2njmzkoxy2n6gg3/fjqtf3hbn35g+ftp6ihbyzr03ucepbnfgdb+dxmli0vjkcjw1tr6xuzxe3inpmfaufwkoqlxxqtpbjssabgsrripb4atw9js3i7/8yjtmobw33yjvamhj3uqujjxq6vmcimrdfqhviggxdscxun9/oigfnifh0o8/5i6isambyk+pejrjzt0x8dzxpisdpiju00pscgkcmgmoak2rnhuzwg+u4vswqyremkvao9bivuslbezxeupfbvjakg3cdju90ucx+rujb4hw3cc3lqgytp71ruj/xju2zytaj8sonkzsyupnwgat4lfkumevo0z0lqgqup0rpm1qqi3nchscn7vypcnlsp6b9e7omvnrarxjtif20shy0dnko1tuqeve0tn6re/ow3lx3pxp52tsmncmpbvod5zhdzr/nzm=</latexit> xlocal (R2 ) <latexit sha1_base64="t6pxhkk2gehiw53zqxezgsfoieo=">aaab63icbvbns8naej3ur1q/qh69lbbbu0lu0gpri8ck9gpaudbbtbt0dxn2j0ij/qtepcji1t/kzx9j0uagrq8ghu/nmdmvikww6lrftmltfwnzq7xd2dnd2z+ohh61bzqyxlsskphpbtryktrvoudju7hhvawsd4ljxe53nrixitkpoi25r+hii1awirnuhfxwbtwaw3fnikvek0gncjqh1a/+mgkj4hqzpnb2pddgp6ugbzn8vuknlseuteii9zkqqelwt+e3zshzpgxjgjmsnjk5+nsipcraqqqytkvxbje9xpzp6yuy3vip0hgcxlpfojcrbcosp06gwncgcporyozibivsta1lmmwth+atv7xk2hd1z617d1e1xm0rrxlo4btowynrama9nkefdmbwdk/w5ijnxxl3phatjaeyoyy/cd5/aaa6jy4=</latexit> <latexit sha1_base64="sfmtfy2viq9fo+d+czscmwtazbs=">aaab63icbvbns8naej3ur1q/qh69lbbbu0leqmeif48v7ae0owy2m3bp7ibstoqs+he8efdeq3/im//gpm1bwx8mpn6bywzeeeth0xw/ndlg5tb2tnm3srd/chhupt7p2cgxjldzjcptc6jlumjerogs92ldqqok7wbtu9zvpnfjraqfcrzzx9gxfqfgfhopnfqqw2rnrbslkhxifaqgbvrd6tdgflfecy1mumv7nhujn1kdgkk+rwwsy2pkpntm+xnvvhhrp4tb5+qiu0ykjexwgslc/t2rumxttavzp6i4satelv7n9rmmb/xu6dhbrtlyuzhighhjhycjythdocsizuzktxi2oyyyzoljq/bwx14nnau659a9h+ta87aiowxnca6x4eedmnaplwgdgwk8wyu8ocp5cd6dj2vryslmtuepnm8fa7cnja==</latexit> <latexit sha1_base64="5vl7yz45j79uamdessxsrgtv2zc=">aaab63icbvbns8naej3ur1q/qh69lbbbu0mkomeif48v7ae0owy2m3bp7ibstoqs+he8efdeq3/im//gpm1bwx8mpn6bywzeeeth0xw/ndlg5tb2tnm3srd/chhupt7p2cgxjldzjcptc6jlumjerogs92ldqqok7wbtu9zvpnfjraqfcrzzx9gxfqfgfhopnwxuhtwaw3cxiovek0gncrsg1a/bkgkj4hqzpnb2ptdgp6ugbzn8xhkklseutemy9zoqqelwtxe3zslfpoxigjmsnjkf+nsipcramqqytkvxyle9xpzp6ycy3vip0hgcxlplojcrbcosp05gwncgcpyryozibivsqg1lmmwth+ctvrxooo2659a9h6ta87aiowxnca6x4me1noeewtagbhn4hld4c5tz4rw7h8vwklpmnmifoj8/btwnjq==</latexit> 2 <latexit sha1_base64="m6whpsvc0ahh0zceaoguhblate0=">aaab63icbvbns8naej3ur1q/qh69lbbbu0mkomeif48v7ae0owy2m3bp7ibstoqs+he8efdeq3/im//gpm1bwx8mpn6bywzeeeth0xw/ndlg5tb2tnm3srd/chhupt7p2cgxjldzjcptc6jlumjerogs92ldqqok7wbtu9zvpnfjraqfcrzzx9gxfqfgfhopnwxuhtwaw3cxiovek0gncrsg1a/bkgkj4hqzpnb2ptdgp6ugbzn8xhkklseutemy9zoqqelwtxe3zslfpoxigjmsnjkf+nsipcramqqytkvxyle9xpzp6ycy3vip0hgcxlplojcrbcosp05gwncgcpyryozibivsqg1lmmwth+ctvrxoold1z617d41a87aiowxnca6x4me1noeewtagbhn4hld4c5tz4rw7h8vwklpmnmifoj8/cd+njw==</latexit> 4 Dictionary elements learned from natural signals with sparse coding may be conceptualized as landmarks on a smooth m nction defined on R2 (e.g. a gray-scale natural image) and one element from its reconstruction are represented by the b

25 Persistence

26 <latexit sha1_base64="s8btnv4fqa19yel5zrbiyd8tbti=">aaab9hicbvbns8naej3ur1q/qh69lbbbqymjcopbkhrrw4smltrp2wy37dlnb7sbpyt+dy8evlz6y7z5b9y0owjrg4hhezpmzpnjzqqyzw+jslk6tr5r3cxtbe/s7px3d5oysgshdol4jno+lpszkdqkku7bsaa48dlt+epbzg89uifzfdbujkzegichgzcclza6tlttulw720dx6l7uk1fmmjkdwizwtiqqw+6vv9x+rjkahopwlgxhmmplpvgorjidltxe0hitmr7sjqyhdqj00tnvu3silt4arejxqnbm/t2r4kdksedrzgcrkvz0mve/r5oowawxsjbofa3jfneg4uhfkisa9zmgrpgjjpgipm9fziqfjkohlyvglb68tjyz2lxnejiv1g/ynipwbmdwchzcqb3uwayhcah4hld4m56mf+pd+ji3fox85hd+wpj8afbqkgi=</latexit> <latexit sha1_base64="6o+ggnzwibydht7xsxvuuj+gogk=">aaab/hicbvdlssnafj34rpuvhzs3g0wom5kioc6eyjcuk5i20iqymu7aofmim5nidcvfcencxa0f4s6/mwm70nydfw7n3mu994qxo9o4zre1tlyyurze2chubm3v7np7+w0te4wjhywtqhuitrgvxdpumnkkfue8zkqzdmq53xwspaku92yuk4cjnqarxchkusc+bfz7pjqpqa84rmnhuixoix275fscceaicwekbgaod+wvvytxwokwmcgt264tmybfyldmyljoj5reca9qj7qzkhanokgn14/hsaz0ysrvvslaifp7ikvc6xeps06otf/pe7n4n9dothqzpfteisectxdfcyngwjwk2kwkymnggufy0exwiptiiwyywpiq3pmxf4l3vrmquhfnpernli0coalhoaxccagq4bbugqcweatp4bw8wu/wi/vufuxbl6zzzah4a+vzb3udlcm=</latexit> <latexit sha1_base64="6o+ggnzwibydht7xsxvuuj+gogk=">aaab/hicbvdlssnafj34rpuvhzs3g0wom5kioc6eyjcuk5i20iqymu7aofmim5nidcvfcencxa0f4s6/mwm70nydfw7n3mu994qxo9o4zre1tlyyurze2chubm3v7np7+w0te4wjhywtqhuitrgvxdpumnkkfue8zkqzdmq53xwspaku92yuk4cjnqarxchkusc+bfz7pjqpqa84rmnhuixoix275fscceaicwekbgaod+wvvytxwokwmcgt264tmybfyldmyljoj5reca9qj7qzkhanokgn14/hsaz0ysrvvslaifp7ikvc6xeps06otf/pe7n4n9dothqzpfteisectxdfcyngwjwk2kwkymnggufy0exwiptiiwyywpiq3pmxf4l3vrmquhfnpernli0coalhoaxccagq4bbugqcweatp4bw8wu/wi/vufuxbl6zzzah4a+vzb3udlcm=</latexit> <latexit sha1_base64="6o+ggnzwibydht7xsxvuuj+gogk=">aaab/hicbvdlssnafj34rpuvhzs3g0wom5kioc6eyjcuk5i20iqymu7aofmim5nidcvfcencxa0f4s6/mwm70nydfw7n3mu994qxo9o4zre1tlyyurze2chubm3v7np7+w0te4wjhywtqhuitrgvxdpumnkkfue8zkqzdmq53xwspaku92yuk4cjnqarxchkusc+bfz7pjqpqa84rmnhuixoix275fscceaicwekbgaod+wvvytxwokwmcgt264tmybfyldmyljoj5reca9qj7qzkhanokgn14/hsaz0ysrvvslaifp7ikvc6xeps06otf/pe7n4n9dothqzpfteisectxdfcyngwjwk2kwkymnggufy0exwiptiiwyywpiq3pmxf4l3vrmquhfnpernli0coalhoaxccagq4bbugqcweatp4bw8wu/wi/vufuxbl6zzzah4a+vzb3udlcm=</latexit> <latexit sha1_base64="6o+ggnzwibydht7xsxvuuj+gogk=">aaab/hicbvdlssnafj34rpuvhzs3g0wom5kioc6eyjcuk5i20iqymu7aofmim5nidcvfcencxa0f4s6/mwm70nydfw7n3mu994qxo9o4zre1tlyyurze2chubm3v7np7+w0te4wjhywtqhuitrgvxdpumnkkfue8zkqzdmq53xwspaku92yuk4cjnqarxchkusc+bfz7pjqpqa84rmnhuixoix275fscceaicwekbgaod+wvvytxwokwmcgt264tmybfyldmyljoj5reca9qj7qzkhanokgn14/hsaz0ysrvvslaifp7ikvc6xeps06otf/pe7n4n9dothqzpfteisectxdfcyngwjwk2kwkymnggufy0exwiptiiwyywpiq3pmxf4l3vrmquhfnpernli0coalhoaxccagq4bbugqcweatp4bw8wu/wi/vufuxbl6zzzah4a+vzb3udlcm=</latexit> <latexit sha1_base64="mn/vnrfrtbcslw3fcifcphm2ave=">aaab7nicbvbns8naej3ur1q/qh69lbbbg5rebd0wvxisyd+gcwwy3brln5uwuxfk6i/w4kerr/4eb/4bt20o2vpg4pheddpzwlrwbvz32ymtrw9sbpw3kzu7e/sh1cojtk4yrvmljijr3ra1e1yylufgsg6qgmahyj1wfdfzo09maz7irznjwrdjupkiuzrw6vjnefcvsf+tuxv3drjkviluoeczx/3ybwnnyiynfah1z3nte+sodkectst+plmkdixd1rnuysx0km/pnzizqwxilchb0pc5+nsix1jrsrzazhjnsc97m/e/r5ez6cbiuuwzwyrdlioyquxczr+tavemgjgxbkni9lzcr6iqgptqxybglb+8stqxdc+tew9xtczteucztuauzsgda2japtshbrtg8ayv8oakzovz7nwswktomxmmf+b8/gc4go8m</latexit> <latexit sha1_base64="zyzbka08naujytw8pm8jf/uv8du=">aaacfnicbvdlssnafj3uv62vqes3g0uqlcupgrqrd8rlbdmwmhgm02k7dgyszizcsfoxbvwvny5u3io7/8b0sddwaxco59zlvfceeankw9a3kvtyxfpeya8w1ty3nrfm7z26cmojiyndfspmgbrhvbbhu81im5ie8ycrrtc/hpmnbyivdcwdhkte46graidipdpjn8sup8jpakoypjw3do6wrtl0vujfwypodhhnyc2nnxqa+nbbn4tw2rodzhn7sopgippvfrntemecci0zuqplw5h2eiq1xywmc26ssirwh3vjk6mccak8zpzxeb5ksht2qpmv0hcs/p5iefdqwioskypdu7pespzpa8w6c+olvesxjgjpfnvibnuiryhbnpueazbicmkszrdc3emsyz1foqrbnn15njiv8lnzvj0uvi+maetbhtghh8agj6akbkanoacdr/amxsgb8ws8go/gx6q1z0xndsefgj8/b1edxg==</latexit> <latexit sha1_base64="hbiqpjxjpmreebr/fuvewpzyvck=">aaab63icbvbnswmxej3ur1q/qh69bivgqcquchosevfywx5au5rsmm1dk+ysziwy9c948aciv/+qn/+n2xyp2vpg4pheddpzwkrwyz3vg5xw1jc2t8rblz3dvf2d6ufr28sppqxfyxhrbkgme1yxluvwsg6igzghyj1wcpf7nsemdy/vo50mljbkphjekbg51oxfkeg15tw9ofaq8qtsgwlnqfwrp4xpkpmyvbbjer6x2caj2niq2kzstw1lcj2qees5qohkjsjmt87wmvogoiq1k2xxxp09krfpzfsgrlmsozblxi7+5/vsg90egvdjapmii0vrkrcncf44hnlnqbvtrwjv3n2k6zhoqq2lp+jc8jdfxixty7rv1f2hq1rjtoijdcdwcufgwzu04b6a0aiky3igv3hder2gd/sxac2hyuyy/gb9/gb7yy3b</latexit> We seek a geometric mapping f : Φ P, s.t. each of the dictionary elements is mapped to a new vector, Pj = f(φj), where Pj is the j th column of P. Continuous temporal transformations in the input should have a linear flow on M and also in the geometrical embedding space. We desire: Pa t 1 2 Pa t Pa t+1 Objective function: s.t. V =Cov(a) D =

27 <latexit sha1_base64="jtdqacsyhdusarjlvigzoythm8o=">aaab7xicbvdlsgnbeoynrxhfuy9ebopgkeykomegf48rzaosjfrojsmy2zllzlyis/7biwdfvpo/3vwbj8kenlggoajqprsrsgq31ve/vcla+sbmvng7tlo7t39qpjxqgpvqyhpucaxberomugqny61g7uqzjcpbwth4dua3npg2xmkho0lygonq8ggnaj3u7kjirtgrv/yqpwdzjufokpcj3it/dfukpjgtlgo0php4iq0z1jztwaalbmpygnsmq9zxvglmtjjnr52sm6f0yubpv9ksufp7ispymekcuc4y7cgsezpxp6+t2sf1mhgzpjzjulg0sawxisxej32ugbvi4ghszd2thi5qi7uuojilivh+ezu0l6qbxw3ulyu1mzyoipzakzxdafdqgzuoqwmopmizvmkbp7wx7937wlqwvhzmgp7a+/wbi4gpga==</latexit> <latexit sha1_base64="f2yzimwbr/dgjzp6tz360fhrqni=">aaab6hicbvbns8naej3ur1q/qh69lbbbu0le0gpri8cw7ae0owy2k3btzhn2n2ij/qvepcji1z/kzx/jts1bwx8mpn6bywzekaiujet+o4w19y3nrej2awd3b/+gfhju0ngqgdzzlglvcahgwsu2dtcco4lcgguc28h4dua3h1fphst7m0nqj+hq8pazaqzueoqxk27vnyosei8nfchr75e/eooyprfkwwtvuuu5ifezqgxnaqelxqoxowxmh9i1vniitz/nd52sm6smsbgrw9kqufp7iqor1pmosj0rnso97m3e/7xuasjrp+mysq1ktlgupokymmy+jgoukbkxsyqyxe2thi2ooszybeo2bg/55vxsuqh6btvrxfzqn3kcrtibuzghd66gbndqhyywqhigv3hzhpwx5935wlqwnhzmgp7a+fwb5jmm/a==</latexit> <latexit sha1_base64="8mzdy9xwepsvigumrvrx3ipm9ue=">aaab7hicbvbns8naej3ur1q/qh69bivgqsqi6lhoxwmfwwttkjvtpf262ytdivbcf4mxd4p49qd589+4bxpq1gcdj/dmmjkxpliy8rxvp7s2vrg5vd6u7ozu7r9ud4/ajsk0xxzpzki7itmohciwczlystwyojt4gi5vz/7je2ojevvakxsdma2viarnzkvwl0ri/wrnq3tzukvel0gncjt71a/eiofzjiq4zmz0fs+ligeabjc4rfqygynjyzberqwkxwicfh7s1d2zyscnem1lkttxf0/kldzmeoe2m2y0msvetpzp62yuxqe5uglgqphiuzrjlxj39rk7ebo5yykljgthb3x5igngyeztssh4yy+vkvzf3ffq/v1lrxftxfggezifc/dhchpwb01oaqcbz/akb45yxpx352prwnkkmwp4a+fzb8pwjqq=</latexit> <latexit sha1_base64="amrqspawr7hi/mdgqkeukpymg8y=">aaachxicdzdnsgmxfiuz/lv/qi7dbisgqgvmhkwigujgzqwrqlplntrtqzozicmipfvf3pgqblwo4skn+dzmtikkhggcvntvknvcrhbtxpfngroegr0bn5jmtu3pzm7l5xdodzwqyio0fre6d0ezwswrgg4eo08ugygu7czshgb1s0umni/liekmrbzbs/imp2asqucdaijpa97dbfslrfzwaznd3o9fbzbym5n16pcvflygibc3nsccxd1fcis721t+sixdouuwpn/ljf8kngpswzkpgayq1/mvpbhtnglsuafavz03mbuekmopync5kmqwao1ai1wtlraxxet9bhenvyxp4gas7jegf9dvez2ito5goe2mwlt171og/6pvu9pcrvw4tfldjp18qjkkbgkcryubxdfqrncaoirbv2labgxu2eczel42xf+bu7/ouuxvocjshwzimeblabmtig+v0d46qmvuqrtdodv0gb6dw+feexkep1uhnmhmivoh5/udeyqgrw==</latexit> The sparse manifold transform

28 Simple example

29 rows that were used in the classical embedding are linear Simple ramp functions and the later example rows have higher frequencies. learned embedding rows of P Figure 3. A functional interpretation of manifold embedding. A)

30 Simple example k-sparse function embedding and recovery 4-sparse function <latexit sha1_base64="midukevp9enlmuwsxqos5lab9xm=">aaab7xicbvbns8naej3ur1q/qh69lbbbu0le0gpri8ck9gpaudbbtbt2kw27e6ge/gcvhhtx6v/x5r9x0+agrq8ghu/nmdmvskqw6lrftmltfwnzq7xd2dnd2z+ohh61juo14y2mpnldgbourcxbkfdybqi5jqljo8hknvc7t1wboeihncbcj+gofqfgfk3u7gccawvqrbl1dw6ysryc1kbac1d96g8vsymei5pumj7njuhnvkngks8q/dtwhlijhfgeptgnupgz+buzcmaviqmvthujmau/jziagtonatszurybzs8x//n6kybxfibijeues8wimjuefclfj0ohoum5tyqyleythi2ppgxtqhki3vllq6r9uffcund/wwvcfhgu4qro4rw8uiig3eetwsdgez7hfd4c5bw4787horxkfdph8afo5w/54o64</latexit> recovery

31 <latexit sha1_base64="olishiyhg4ag1upjk4yi4gtsjye=">aaab7nicbvdlsgnbeoz1gemr6thlyba8hv0r9bj04jgceucyhn7jjbkyozvmzaphyud48aciv7/hm3/jblihtsxokkq66e6klodg+v63t7a+sbm1xdop7+7thxxwjo5bjkk1zu2aier3ijrmcmmallvbokozjcpb2thklvfbt0wbnshho1usjhek+zbtte5q91comzb7lapf8+cgqyqosbuknpqvr94gownmpkucjekgvrjhhtpyktis3esnu0gnogjdryxgzitz/nwzoxfkgawt7upamld/t2qygzoni9czox2bzs8x//o6qr3ehbmxkrvm0swiysqituj+oxlwzagvu0eqau5ujxsmgql1ceuhbmsvr5lwzs3wa8hdvbv+w8rrglm4gwsi4brqca8naakfctzdk7x5ynvx3r2preuav8ycwb94nz/b+y8s</latexit> <latexit sha1_base64="au2pysm3bhfn5ul25fiumeti3uw=">aaab6xicbvbns8naej3ur1q/qh69lbbbu0le0gpri8cqthbaudbbtbt0swm7e7ge/gmvhhtx6j/y5r9x0+agrq8ghu/nmdmvskqw6lrftmlldw19o7xz2dre2d2r7h+0tzxqxlsslrhubnrwkrrvoudjo4nmnaokfwjg17n/8mi1ebg6x0nc/ygolqgfo2ilu6dkv1pz6+4mzjl4balbgwa/+tubxcynueimqtfdz03qz6hgwssfvnqp4qllyzrkxusvjbjxs9mlu3jilqejy21lizmpvycyghkziqlbgvecmuuvf//zuimgl34mvjiiv2y+kewlwzjkb5ob0jyhnfhcmrb2vsjgvfognpw8bg/x5wxspqt7bt27pa81roo4ynaex3akhlxaa26gcs1gemizvmkbm3zenhfny95acoqzq/gd5/mhgsynea==</latexit> <latexit sha1_base64="uupgqesxuoeaqokskj/yzfwqzk8=">aaab73icbvbns8naej34wetx1aoxxsj4kokieix68vjbfkabyma7azdunnf3ipbqp+hfgyje/tve/ddu2hy09cha470zzuyfirqgxffbwvldw9/ylg2vt3d29/yrb4cte6ea8salzaw7atvccswbkfdytqi5jqlj28h4jvfbj1wbeat7nctcj+hqivawilbq9eyus6dpuv+pujv3brjmvijuoucjx/nqdwkwrlwhk9syrucm6gduo2cst8u91pcesjed8q6likbc+nns3ik5tcqahlg2pzdm1n8tgy2mmusb7ywojsyil4v/ed0uwys/eypjkss2xxsmkmbm8ufjqgjoue4souwleythi6opqxtrhok3+piyaz3xplfm3v1u69dfhcu4hhm4aw8uoq630iammjdwdk/w5jw4l8678zfvxxgkmsp4a+fzb+ddj90=</latexit> <latexit sha1_base64="midukevp9enlmuwsxqos5lab9xm=">aaab7xicbvbns8naej3ur1q/qh69lbbbu0le0gpri8ck9gpaudbbtbt2kw27e6ge/gcvhhtx6v/x5r9x0+agrq8ghu/nmdmvskqw6lrftmltfwnzq7xd2dnd2z+ohh61juo14y2mpnldgbourcxbkfdybqi5jqljo8hknvc7t1wboeihncbcj+gofqfgfk3u7gccawvqrbl1dw6ysryc1kbac1d96g8vsymei5pumj7njuhnvkngks8q/dtwhlijhfgeptgnupgz+buzcmaviqmvthujmau/jziagtonatszurybzs8x//n6kybxfibijeues8wimjuefclfj0ohoum5tyqyleythi2ppgxtqhki3vllq6r9uffcund/wwvcfhgu4qro4rw8uiig3eetwsdgez7hfd4c5bw4787horxkfdph8afo5w/54o64</latexit> Encoding of a natural video sequence

32 Stacked Sparse Manifold Transform layer 3 layer 2 layer 1

33 <latexit sha1_base64="sld2tr8g7jc72ycmid61qq6ybwm=">aaab83icbvbns8naej34wetx1aoxxslus0lu0gpri8ck9goawdbbabt0swm7g6ge/g0vhhtx6p/x5r9x2+agrq8ghu/nmdmvtatxxnw/nzxvtfwnzcjwcxtnd2+/dhdy1hgqgdzylglvdqlgwsu2ddcc24lcgoucw+hoduq3nlbphsshm04wioha8j5n1fjj9+td/phvls4m3vg3vhar7gxkmxg5kuooerf05fdilkyodrnu647njibiqdkcczwu/vrjqtmidrbjqaqr6icb3twhp1bpkx6sbeldzurviyxgwo+j0hzg1az1ojcv//m6qelfbxmxswpqsvmifiqiick0anljcpkry0sou9zestiqksqmjaloq/awx14mzfoq51a9+8ty7sapowdhcaiv8oakanahdwgagwse4rxennr5cd6dj3nrippphmefoj8/h++rew==</latexit> <latexit sha1_base64="f2yzimwbr/dgjzp6tz360fhrqni=">aaab6hicbvbns8naej3ur1q/qh69lbbbu0le0gpri8cw7ae0owy2k3btzhn2n2ij/qvepcji1z/kzx/jts1bwx8mpn6bywzekaiujet+o4w19y3nrej2awd3b/+gfhju0ngqgdzzlglvcahgwsu2dtcco4lcgguc28h4dua3h1fphst7m0nqj+hq8pazaqzueoqxk27vnyosei8nfchr75e/eooyprfkwwtvuuu5ifezqgxnaqelxqoxowxmh9i1vniitz/nd52sm6smsbgrw9kqufp7iqor1pmosj0rnso97m3e/7xuasjrp+mysq1ktlgupokymmy+jgoukbkxsyqyxe2thi2ooszybeo2bg/55vxsuqh6btvrxfzqn3kcrtibuzghd66gbndqhyywqhigv3hzhpwx5935wlqwnhzmgp7a+fwb5jmm/a==</latexit> <latexit sha1_base64="zinnqoh2wynwzsx2+1u95+kfqfc=">aaab9hicbvdlsgnbeoz1gemr6thlybdijeykomegf48rzaosncxozpmhsw9neonh2e/w4kerr36mn//gsbihtsxokkq66e7yyik02va3tbk6tr6xwdgqbu/s7u2xdg6bokou4w0wyui1paq5fcfvoedj27hinpakb3mjm6nfgnolrrte4ytmbkahofafo2gktzukmd5ld2nfoct6pbjdtwcgy8tjsrly1hulr24/yknaq2ssat1x7bjdlcoutpks2e00jykb0qhvgbrsggs3nr2dkvoj9ikfkvmhkpn6eyklgdatwdodacwhxvsm4n9ej0h/yk1fgcfiqzzf5cesyesmczc+ujyhnbhcmrlmvskgvfggjqeiccfzfhmznm+rjl117i7ktes8jgicwwluwiflqmet1kebdb7hgv7hzrpbl9a79tfvxbhymsp4a+vzb3crkd0=</latexit> <latexit sha1_base64="ifgdmsi+ykpoysbhtqdrbgv1tfi=">aaab83icbvbns8naej34wetx1aoxxslus0lu0gpri8ck9goawdbbabt0swm7g6ge/g0vhhtx6p/x5r9x2+agrq8ghu/nmdmvtatxxnw/nzxvtfwnzcjwcxtnd2+/dhdy1hgqgdzylglvdqlgwsu2ddcc24lcgoucw+hoduq3nlbphsshm04wioha8j5n1fjj9+td/phvls4mxdetld2qownzjl5oypcj3i19+b2yprfkwwtvuuo5iqkyqgxnaidfp9wyudaia+xykmmeoshmn0/iqvv6pb8rw9kqmfp7iqor1umotj0rnuo96e3f/7xoavrxqczlkhqubl6onwpiyjingps4qmbe2blkfle3ejakijjjyyraelzfl5dj87zquvxv/rjcu8njkmaxneafplicgtxbhrraiifneiu3j3venhfny9664uqzr/ahzucpixorfa==</latexit> <latexit sha1_base64="1k82nwtdeqjr9lega9ybtbijwzu=">aaacghicbvbns8naej3ur1q/oh69bitqewpsbt0wvxisyd+giwwz3brln5uwuxfkym/w4l/x4kerr735b9y2owjtg4hhezpmzpnjrqwy7w+jslk6tr5r3cxtbe/s7pn7by0zjqktjo5yjdo+korrtpqkkky6ssao9blp+6pbqd9+iklsid+ocuy8ea04dshgsks989wnbmkpk6w1zg0m6wnautjneqoz5trrmww7as9g/sdotsqqo9ezj24/wkliumimsdl17fh5krkkykaykptieim8qgps1zsjkegvnt2wwsda6vtbjhrxzc3u3xmpcquch77udjeaykvvki7zuokkrr2u8jhrhop5oibhloqsaupwnwqcfrtrgrcg+lyld5foroksszoez/hl/6rvqzp21bm/lndv8jikcathuaehrqaod9cajmb4hld4hw/jxxgzpo2vewvbygco4q+myq+u0z+f</latexit> <latexit sha1_base64="xdqxkquukpbdd2c/afadhdhvwhg=">aaab9hicbvbns8naej3ur1q/qh69bitqlyupgh6lxjxwsb/qxrlzbtulm03cnrrlyo/w4kerr/4yb/4bt20o2vpg4pheddpz/ehwjy7zbexw1jc2t/lbhz3dvf2d4ufru4exoqxbqxgqtk80e1yybniurb0prgjfsjy/vpn5rqltmofyhqcr8wiylhzakuejed0rweqpfujk1fo0vyw5fwcoe5w4gslbhnqv+nxthzqomeqqinyd14nqs4hctgvlc91ys4jqmrmyjqgsbex7yfzo1d4zst8ehmqurhuu/p5iskd1npbnz0bwpje9mfif14lxcoulxeyxmkkxiwaxsdg0zwnyfa4yrte1hfdfza02hrffkjqcciyed/nlvdksvlyn4t5dlgrxwrx5oiftkimll1cdw6hdayg8wjo8wps1sv6sd+tj0zqzsplj+apr8wdyf5he</latexit> <latexit sha1_base64="th+grho07f4emvazxgdptjkv3uc=">aaab9hicbvdjsgnbek2jw4xb1koxxides5hrqy9blx4jmawsmfr0epimpyvdncewzhd48acivz/gm39jzzlo4oocx3tvvnxzyik02va3lvtzxvvfyg8wtrz3dvek+wcnhswk8tqlzkrahtvcipdxuadkrvhxgnisn73hzcrvjrjsigrvcrxzn6d9upicutss2xlqtj+yh7r8fpp1iyw7yk9blokzjywyo9ytfnv6euschiktvou2y8foplshyjjnhu6ieuzzkpz529cqbly76ftojjwypuf8sjkkkuzv3xmpdbqeb57pdcgo9ki3ef/z2gn6v24qwjhbhrlzij+rbcmysyd0hoim5dgqypqwtxi2oioyndkvtajo4svlphfwceykc3drql7p48jderxdgry4hcrcqg3qwoarnuev3qyr9wk9wx+z1pw1nzmep7a+fwbznzhf</latexit> Stacked Sparse Manifold Transform reconstruction from layers 1,2,3 flattening at layer 3

34 ... DiCarlo & Cox (2007)

35 Main points Sparse coding provides a foundation for manifold learning. A geometrical embedding of the dictionary may be learned by exploiting temporal persistence of structure in the visual world. Manifold flattening may be accomplished in a progressive manner by successive stages of sparse coding (dimensionality expansion) and linear projection (dimensionality collapse) in the ventral stream of visual cortex.