Intro to Deep Learning for Computer Vision
|
|
- Spencer Allen
- 5 years ago
- Views:
Transcription
1 High Level Computer Vision Intro to Deep Learning for Computer Vision Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de
2 Overview Today Recent successes of Deep Learning necessarily incomplete and biased :-) What is Deep Learning? Neural network basics Supervised learning with error backpropagation [reading Bishop chapter ] Convolutional Neural Networks "2
3 Computer Vision: Image Classification 1000 classes; 1ms for an image; 92.5% recognition rate "3
4 Architectures get deeper Top 5 error More accurate caffe-alex vgg-f vgg-m googlenet-dag vgg-verydeep-16 resnet-50-dag resnet-101-dag resnet-152-dag caffe-alex vgg-f vgg-m googlenet-dag vgg-verydeep-16 resnet-50-dag resnet-101-dag resnet-152-dag ~ 2.6x improvement in 3 years "4
5 Architectures get deeper "5
6 Architectures get deeper "6
7 Computer Vision: Semantic Segmentation Badrinarayanan et al SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling "7
8 Computer Vision: Semantic Segmentation Input Groundtruth Output Yang He; Wei-Chen Chiu; Margret Keuper; Mario Fritz STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017 "8
9 Deep(er)Cut: Joint Subset Partition & Labeling for Multi Person Pose Estimation DeepCut: Joint Subset Partition and Labeling for Multi-Person Pose Estimation L. Pishchulin, E. Insafutdinov, S. Tang, B. Andres, M. Andriluka, P. Gehler, and B. Schiele, CVPR 16 DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model E. Insafutdinov, L. Pishchulin, B. Andres, M. Andriluka, and B. Schiele, ECCV 16
10 Analysis - overall performance Best Methods now: deep learning takes over PCKh total, MPII Single Person Best Method as of ICCV 13 since CVPR 14, dataset has become de-facto standard benchmark large training set facilitated development of deep learning methods Progress in Person Detection and Human Pose Estimation in the last Decade Bernt Schiele "10
11 Predicting the Future Apratim Bhattacharyya; Mateusz Malinowski; Bernt Schiele; Mario Fritz Long-Term Image Boundary Extrapolation arxiv: [cs.cv] & AAAI 18 "11
12 Computer Vision & NLP: Image Captioning Ours: a person on skis jumping over a ramp Ours: a skier is making a turn on a course Ours: a cross country skier makes his way through the snow Ours: a skier is headed down a steep slope Baseline: a man riding skis down a snow covered slope Rakshith Shetty; Marcus Rohrbach; Lisa Anne Hendricks; Mario Fritz; Bernt Schiele Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training arxiv: [cs.cv], ICCV 2017 "12
13 Computer Vision + NLP: Visual Turing Test What is on the refrigerator? What is the colour of the comforter? What is the color of the comforter? How many drawers are there? magnet, paper blue, white 3 Mateusz Malinowski; Mario Fritz A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input Neural Information Processing Systems (NIPS), Mateusz Malinowski; Marcus Rohrbach; Mario Fritz Ask Your Neurons: A Neural-based Approach to Answering Questions about Images IEEE International Conference on Computer Vision (ICCV), 2015, "13
14 AI and Planning "14
15 Robotics "15
16 Creative Aspects deepart.io "16
17 Imitating famous painters UVA DEEP LEARNING COURSE EFSTRATIOS GAVVES DEEPER INTO DEEP LEARNING AND OPTIMIZATIONS 19-19
18 Privacy & Safety Seong Joon Oh; Mario Fritz; Bernt Schiele Adversarial Image Perturbation for Privacy Protection - A Game Theory Perspective arxiv: [cs.cv], ICCV Tribhuvanesh Orekondy; Bernt Schiele; Mario Fritz Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images arxiv: [cs.cv], ICCV "18
19 Overview Today Recent successes of Deep Learning necessarily incomplete What is Deep Learning? Neural network basics Supervised learning with error backpropagation [reading Bishop chapter ] Convolutional Neural Networks "19
20 Deep Learning Deep Learning is based on Availability of large datasets Massive parallel compute power Advances in machine learning over the years Strong improvements due to Internet (availability of large-scale data) GPUs (availability of parallel compute power) Deep / hierarchical models with end-to-end learning "20
21 Overview of Deep Learning "21
22 x<latexit sha1_base64="f2 y<latexit sha1_base64="l2 Traditional Approach feature extraction (hand crafted) Classification Car Feature extraction often hand crafted and fixed might be too general (not task-specific enough) might be too specific (does not generalize to other tasks) How to achieve best classification performance more complex classifier (e.g. multi-feature, non-linear)? how specialized for the task? "22
23 Features Alone Can Explain (Almost) 10 Years of Progress workshop 14] Progress in Person Detection and Human Pose Estimation in the last Decade Bernt Schiele "23
24 Motivation Features are key to recent progress in recognition Multitude of hand-designed features currently in use SIFT, HOG, LBP, MSER, Color-SIFT. Where next? Better classifiers? Or keep building more features? Felzenszwalb, Girshick, McAllester and Ramanan, PAMI 2007 Yan & Huang (Winner of PASCAL 2010 classification competition) slide credit: Rob Fergus
25 Hand-Crafted Features LP-β Multiple Kernel Learning (MKL) Gehler and Nowozin, On Feature Combination for Multiclass Object Classification, ICCV different kernels PHOG, SIFT, V1S+, Region Cov. Etc. MKL only gets few % gain over averaging features à Features are doing the work slide credit: Rob Fergus
26 x<latexit sha1_base64="f2 <latexit sha1_base64="qg0sjzvwmzsqdenusdi4/2+vvru=">aaacknicbvdlsgmxfm34rpvvdekmwiqkpcz0ge1cqlhxwce+ob1kjr3thmyejbmhdp0en/6kmy6u4typmx2jwg8edufck9wcj+rmktocghubw9s7u4m95p7b4dfx6us0iyniukjtgaei5rajnplqv0xxaiuciodwadrdu5nffaihwea/qleitkf6pnmzjupl3drt3jlf0hz9x47nxkvuyhclwu3khxzfzfo5c44xhueb7gzcicosilo4e3jqj1fjbiq9iufvdk9i3ysnlqh1u5nol6crb y<latexit sha1_base64="l2 Deep Learning: Trainable features Features features = g(x, ) Classifier y = f(features,!) Car Parameterized feature extraction Features should be efficient to compute efficient to train (differentiable) "26
27 x<latexit sha1_base64="f2 <latexit sha1_base64="qg0sjzvwmzsqdenusdi4/2+vvru=">aaacknicbvdlsgmxfm34rpvvdekmwiqkpcz0ge1cqlhxwce+ob1kjr3thmyejbmhdp0en/6kmy6u4typmx2jwg8edufck9wcj+rmktocghubw9s7u4m95p7b4dfx6us0iyniukjtgaei5rajnplqv0xxaiuciodwadrdu5nffaihwea/qleitkf6pnmzjupl3drt3jlf0hz9x47nxkvuyhclwu3khxzfzfo5c44xhueb7gzcicosilo4e3jqj1fjbiq9iufvdk9i3ysnlqh1u5nol6crb y<latexit sha1_base64="l2 Deep Learning: Joint Training of all Parameters Features features = g(x, ) Classifier y = f(features,!) Car Parameterized feature extraction Features should be efficient to compute End-to-End System efficient to train (differentiable) Joint training of feature extraction and classification Feature extraction and classification merge into one pipeline "27
28 x<latexit sha1_base64="f2 <latexit sha1_base64="qg0sjzvwmzsqdenusdi4/2+vvru=">aaacknicbvdlsgmxfm34rpvvdekmwiqkpcz0ge1cqlhxwce+ob1kjr3thmyejbmhdp0en/6kmy6u4typmx2jwg8edufck9wcj+rmktocghubw9s7u4m95p7b4dfx6us0iyniukjtgaei5rajnplqv0xxaiuciodwadrdu5nffaihwea/qleitkf6pnmzjupl3drt3jlf0hz9x47nxkvuyhclwu3khxzfzfo5c44xhueb7gzcicosilo4e3jqj1fjbiq9iufvdk9i3ysnlqh1u5nol6crb y<latexit sha1_base64="l2 Deep Learning: Joint Training of all Parameters Features features = g(x, ) Classifier y = f(features,!) Car End-to-End System All parts are adaptive No differentiation between feature extraction and classification Non linear transformation from input to desired output "28
29 x<latexit sha1_base64="f2 <latexit sha1_base64="qg0sjzvwmzsqdenusdi4/2+vvru=">aaacknicbvdlsgmxfm34rpvvdekmwiqkpcz0ge1cqlhxwce+ob1kjr3thmyejbmhdp0en/6kmy6u4typmx2jwg8edufck9wcj+rmktocghubw9s7u4m95p7b4dfx6us0iyniukjtgaei5rajnplqv0xxaiuciodwadrdu5nffaihwea/qleitkf6pnmzjupl3drt3jlf0hz9x47nxkvuyhclwu3khxzfzfo5c44xhueb7gzcicosilo4e3jqj1fjbiq9iufvdk9i3ysnlqh1u5nol6crb y<latexit sha1_base64="l2 Deep Learning: Complex Functions by Composition Features features = g(x, ) Classifier y = f(features,!) Car End-to-End System How can we build such systems? What is the parameterization (hypothesis)? Composition of simple building blocks can lead to complex systems (e.g. neurons - brain) "29
30 x<latexit sha1_base64="f2 y<latexit sha1_base64="l2 Deep Learning: Complex Functions by Composition Car How can we build such systems? What is the parameterization (hypothesis)? Composition of simple building blocks can lead to complex systems (e.g. neurons - brain) each block has trainable parameters Each block has trainable parameters i "30
31 x<latexit sha1_base64="f2 y<latexit sha1_base64="l2 Deep Learning: Complex Functions by Composition Car intermediate representations How can we build such systems? What is the parameterization (hypothesis)? Composition of simple building blocks can lead to complex systems (e.g. neurons - brain) each block has trainable parameters Each block has trainable parameters i "31
32 x<latexit sha1_base64="f2 y<latexit sha1_base64="l2 Deep Learning: Complex Functions by Composition Car Lee et al. Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations "32
33 x<latexit sha1_base64="f2 y<latexit sha1_base64="l2 Deep Learning: Complex Functions by Composition Car Lee et al. Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations "33
34 x<latexit sha1_base64="f2 y<latexit sha1_base64="l2 Deep Learning: Complex Functions by Composition Car Lee et al. Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations "34
35 Summary of Main Ideas in Deep Learning 1. Learning of feature extraction (across many layers) 2. Efficient and trainable systems by differentiable building blocks 3. Composition of deep architectures via non-linear modules 4. End-to-End training: no differentiation between feature extraction and classification "35
36 Summary of Main Ideas in Deep Learning 1. Learning of feature extraction (across many layers) 2. Efficient and trainable systems by differentiable building blocks 3. Composition of deep architectures via non-linear modules 4. End-to-End training: no differentiation between feature extraction and classification "36
37 Overview Today Recent successes of Deep Learning necessarily incomplete What is Deep Learning? Neural network basics Supervised learning with error backpropagation [reading Bishop chapter ] Convolutional Neural Networks "37
38 Some Inspiration from the (Human) Brain Motivation from the brain effective universal learning machine? Highly simplified neural model Combination of inputs is linear No spike No temporal model Fixed topology Properties we want: differentiable efficient "38
39 <latexit sha1_base64="bt0wvb2/c4ych9uvc+nkbx5n5uq=">aaab+hicbvdlssnafl2pr1ofjbp0m1gef6ukiuhkcrpwwce+oa1hmp20qyetmdor1tavcencebd+ijv/xmmbhbyegdiccw/3zgkszpr2ng+rsla+sblv3c7t <latexit sha1_base64="ip0vb2yartlwduubaa6y x<latexit sha1_base64=" <latexit sha1_base64="t9knuvgwh0bdbnc+u4o54setwds=">aaab8hicbvbnswmxej31s9avqkcvwslus9kvqrgughepfeyhtevjptk2nmkusvzyl/4klx4u8erp8ea/mw33ok0pbh7vztazl4g z<latexit sha1_base6 <latexit sha1_base64= [Reading - Chapter Bishop 2006] Neural Network Basics Core component of a neural network: processing unit ~ neuron of the human brain a processing unit maps multiple input values onto one output value y<latexit sha1_base64= x 0,...,x D are input units w 0 is an external input called bias the propagation rule maps all input values on the actual input the activation function is applied to obtain: y := f(z) slide adapted from David Stutz "39
40 <latexit sha1_base64="8zyp5zxohwzp73ybmhg8do61faa="> <latexit sha1_base64="yvg82ih+xozcgc0czea6whqqg98=">aaacihicbvbns8naen34wetx1aoxwsj4kcurqfegqi8ek9gqnkvstpt2cbmjuxm1hp4ul/4vlx4u0zv+grc1b7u+ghi8n7m784jecoou++hmzm7nlyywlsrlk6tr65wnzbaju814i8uy1lcbnvwkxvsoupkrrhmabzjfbtensx95w7ursbralohdia6ucawjakve5vacq9yt4edbfjcd3aocaozakkkbl3mifu7v/h6mpgyn8luydnef9spvt+5oanpek0ivfgj2ku/2dzzgxcgt1jio5ybyzalgwsqflf3u8isyazrghusvjbjp5pmdr7brlt6esbalecbqz4mcrszkuwa7i4pd89cbi/9 <latexit sha1_base64="cgd6hc9y5yxoqh7kwfta8eyimfu=">aaacxxicbvfns8mwge7r9/yaevdg5cuhtjtriqaxrdcdrwwnwjplmqvbzvpb8ladpx/sm178k2a1b52+ehjyfjdkszbkodfx3i17anpmdm5+oba4tlyywl9bv9vjphhvs0qm6j6gmksr8zyklpw+vzxgger3wep5wl974kqljl7bucq7ee3hihsmoqh8oo58au142ydn2iutcjuvvigbejkh2arpz5gfd0/c4uecnv1cdat48yewv26cajwl+gp8n+nmzcqrx284lacc+avccjrinvd+/c3rjsyleixmuq07rpnin6ckbzo8qhmz5illj7tpowbgnok6m5ftflbjmb6eitirrijzn4mcrlqposa4i4odpamnyf+0tobhctcxczohj9n3qwemarmyvw09othdotkamixmxyenqkimzyfutanu5jp/gtudluu03ovdxtlpvcc82slbpelccktoycw5im3cyidfrawrzn3am/asvfjtta0qs0f+jb35br8rsbw=</latexit> i<latexit sha <latexit sha1_base64="z0ragrrbgeijgv <latexit sha1_base64="wtmrxvv1afs/uirgycbzxwafe4i=">aaacd3icbvdlssnafj3uv62vqes3g0urhjkiojtkqrcuk9ghtdfmppn22pkkzezugvihbvwvny4ucevwnx/jtm1cww9cohpovcy9x4sylcqyvo3c3pzc4lj+ubcyura+yw5u1wuyc0xqogshahpiekydulnumdkmbehcy6thdc5hfuowcend4foni+jw1a2otzfswnln/qexwjjsy5i7sb9spzcx8m5na <latexit sha1_base64="mtycvzzyxl <latexit sha1_base64="6eznfk1ek25sns <latexit sha1_base64="sh7gpcp8wxkv12c <latexit sha1_base64="eyrjlu/z1pz/6xn7koj <latexit sha1_base64="hp+6lruf2d3tzaldqaqqvekmxyw=" sha1_base64="c9rktwtrme5pezcp4bi4eewdoc4=" sha1_base64="vwmiait3qcyv3r6mnx5qxr1z2fa=" sha1_base64="hbtzlwl23bixrhv7kydwsfqoagk=" Neural Network Basics: Perceptron [Rosenblatt 1958] A single-layer perceptron consists of D input units and C output units input unit j : x j with j 2 {0,...,D}(x 0 := 1) propagation rule: weighted sum over inputs with weights value and additional bias w i0 DX z i = w ij x j + w i0 activation function: next slide output unit calculates output for class j=1 i : y i with i 2 {1,...,C} x j w 10 w C0 w ij w 1D w CD 0 1 DX y i (x, w) =f(z i )=f@ w ij x j + w i0 A = f j=1 0 1 w ij x j A j=0 slide adapted from David Stutz "40
41 Short Intro: Perceptron - Activation Functions slide taken from David Stutz "41
42 Single Layer Perceptron slide taken from David Stutz "42
43 Two-Layer Perceptron slide taken from David Stutz "43
44 l<latexit sh <latexit sha1_base64="biagji/gfszf20aqsz965gp/1ko=">aaab7nicbvbnsw i<latexit sha <latexit sha1_base64="m3py6/pva2qknlnzgofyvjzy5v4=">aaab8nicbvbnswmxej2tx7v+vt16crahxsquchprcnrwwmf+whyt2ttbhmatjckkzenp8ojbea/+gm/+g9n2d9r6yodx3gwz88kem21c99s l<latexit <latexit sha1_base64="5sfe1kkt6rzsezt3bso2rwcrf90=">aaacnxicbvdlssnafj34rpvvdelmsajtwpkiobtfcopcrqwrqlpdzdppp51jwsynekj+yo3/4uoxlhrx6y84rrf8hrg4nhmud+7xy8e12pajnte5nt0zw5orzy8sli1xvlbpdzqoylo0epg69ilmgoesbrweu4wvi9ix7mifho38i2umni/cm0hj1pgkf/kauwjg8ionqcevspqo53gfb9gvliaadnuivwywb+dxmrzzw049z/gnl/fbxsrtb/dlyffxxh/qxqvqn+wx8f/ifkskcjs9yr3bjwgiwqhuek3bjh1djymkobusl7ujzjghq9jjbundi <latexit sha1_base64="drjec8awsn+nsha0nudnp6wvlra=">aaacgnicbvdlsgnbejynrxhfqx69dayhgorderqpeoghj1hma7ixze4myzdznwwm1xcwficxf8wlb0w8irf/xsnjebmlgoqqbrq7/ehwdy7zy6wwlldw19lrmy3nre0de3evomwskcttkasq+uqzwunwbg6c1slfsoalvvv7xzfffwrkcxnewybijyb0qt7mlicrmry7yhm0jegkf3yjvyba1/etu+hdnfyu73sbkcx7s <latexit sha1_base64="lftn0prylzmqf6zriwchflmjqja=">aaab9hicbvbnswmxej2tx7v+vt16crahiprdefsifhrx4kgc/yb2ldk0buot7jpkc2xp7/diqrgv/hhv/hvtdg/a+mdg8d4mm/ocidntxpfbyaysrq1vzddzw9s7u3v5/yo6dmnfai2epftnagvkma <latexit sha1_base64="gobhlay4gk0oz6a8weqv0zxcycq=">aaacfnicbvhlssnafj3ed31vxbozlup91uqeu1ekbly4ulaqndvmjrft4gqszizqcp0mf8yd3+lgartqaw8mhm65r7k3sdht2ne+lxtqemz2bn6htli0vljaxlu/v3eqktrpzgp5gbafnaloaqy5pcyssbrweaielwf+wwtixwjxp7me2hhpctzhlggj+ex3rpp2+lqhz7exqjejpimiluytjzpfhvmeh72xmnzqwdl/slbbhd/be9of8ur1gbvxn1xmzp1tzi9xnjozbp5p3ijuuiebv/zhhtfnixcacqjuy3us3c6j1ixy6je8vefc6dppqstqqsjq7xy4vj7emuqio7e0t2g8vh9n5crskosce2kg7klxbybo8lqp7ttbornjqkhquanoyrgo8eawogqsqoaziyrkzmbfteckodpcrgsw4i5/+t+5p6m5ts29pa00lop1zknnti2qyevnqigu0a1qioq+rc1r3zqwkb1rh9nho1dbkni20b/y9w84571q</latexit> Multi-Layer Perceptron (MLP) add additional L > 0 hidden layers in between the input and output layers m (l) m (0) = D m (L+1) = C layer has units with and then the output of unit in layer is given by: y (l) i = f m (l 1) X j=0 w (l) ij y(l 1) j 1 A a multi-layer perceptron models a function: y(,w):r D! R C y(x, w) = 0 1 y 1 (x, w) y C (x, w) C A = (x, w) C. A C (x, w) y (L+1) y (L+1) slide adapted from David Stutz "44
45 x<latexit sha1_base64="f2yzimwbr/dgjz y<latexit sha1_base64="l29wxoub9d Training: Overview Car? Setting generate output y for input x (forward pass) "45
46 x<latexit sha1_base64="f2yzimwbr/dgjz y<latexit sha1_base64="l29wxoub9d Training: Overview Error? Setting generate output y for input x (forward pass) when there is an error, propagate error backwards to update weights (error back propagation)
47 Network Training slide taken from David Stutz "47
48 <latexit sha1_base64="lbhufbfzeo+wstvcxlb/mie6vhq=">aaacu3icbvfdsxtbfj1du7wxarspfbk0cbhasbsefrfeexwqkzheymzldjkbjm7oljn3w8oy/1gepvhhfpghnst74ecvdjx77jncmtnrjovbz3t03jxah9w19y/1ju+bw9unnd2+sxpnei+lmtvxetvccsv7kfdyq0xzmkssd6lbs/l88itri1j1ibomjxi6usiwjkklwsbneev3phxdypiklnsxx17/gpnqvwysksv8suiurxxlrsybmwgkj7efs1c07kl1daz1o6dvihig0giyxf3rdosnptf2fgxvgv+bjqmqgzyegnhk8oqrzjiam/s9decf1siy5gu9ya3pklulez60ungem1gxyksepcumiu61pqphwb50fdqxzpzevplqnjq3szn5v9kwx/hovaiv5cgvwy6kcwmywjxggavngcqzbzrpye8kbeptjmi/ow5d8n8++t3od9q+1/z/hjrptqs41skx8pw0ie8oyqm5if3si4zckyfy1yhoh+fzdd3auuo6leczevxu5j+fj6+q</latexit> <latexit sha1_base64="7kynmkdp7c6elhiyibpi+qyvzcu=">aaacoxicbzblswmxemez9vxra9wjl2arwtcyk4jecsvs8cqv7apadcmm2ty0m12srlus/vpe/bbebc8efphqfzb9hgx1ipcf38wwmb8xmsqvzb0yqaxlldw19hpmy3nre8fc3avlmbay1hdiqth0kcsmcljtvdhsjarbgcdiw+uxx/xgprgshvxwdspibkjlqu8xuhq5zrwsg+rhebzlhlgjl9qju2tycfmunszzaulhsrkqjekitlnyhuox5h5cfgwhedfmwgvrevcvsgcic2zrdc3ndifecuc4wgxj2bktsdkjeopirkazdixjhhafdulls44cip1kcvkihmnsgx4o9omktujviqqfug4dt3cgspxkym0m/6u1yuvfoanluawix9nffsygcuhyrtihgmdfhloglkj+k8q9jbbw2uymnsfeppmvqj8wbktg35xls5czo9lgabychldbosibk1afnydbi3gf7+ddedleje/ja9qammyz+2aujo8f8y6qew==</latexit> Network Training - Error Measures Sum-of-squared error function: E(w) = NX E n (w) = 1 2 n=1 weight vector NX n=1 CX (y i (x n,w) t ni ) 2 i=1 i-th component of function y i-th entry of target t Cross-entropy error function NX NX CX E(w) = E n (w) = t ni log y i (x n,w) n=1 n=1 i=1 slide adapted from David Stutz "48
49 <latexit sha1_base64="mselxol8+csqoz9dzmua0i0z984=">aaab7hicbvbns8naej34wetx1aoxxslus0 <latexit sha1_base64="lbhufbfzeo+wstvcxlb/mie6vhq=">aaacu3icbvfdsxtbfj1du7wxarspfbk0cbhasbsefrfeexwqkzheymzldjkbjm7oljn3w8oy/1gepvhhfpghnst74ecvdjx77jncmtnrjovbz3t03jxah9w19y/1ju+bw9unnd2+sxpnei+lmtvxetvccsv7kfdyq0xzmkssd6lbs/l88itri1j1ibomjxi6usiwjkklwsbneev3phxdypiklnsxx17/gpnqvwysksv8suiurxxlrsybmwgkj7efs1c07kl1daz1o6dvihig0giyxf3rdosnptf2fgxvgv+bjqmqgzyegnhk8oqrzjiam/s9decf1siy5gu9ya3pklulez60ungem1gxyksepcumiu61pqphwb50fdqxzpzevplqnjq3szn5v9kwx/hovaiv5cgvwy6kcwmywjxggavngcqzbzrpye8kbeptjmi/ow5d8n8++t3od9q+1/z/hjrptqs41skx8pw0ie8oyqm5if3si4zckyfy1yhoh+fzdd3auuo6leczevxu5j+fj6+q</latexit> w<latexit sha1_base64="gme7o Learning by Optimization Highly non-convex error landscape E(w) E(w) = NX E n (w) = 1 2 n=1 NX n=1 CX (y i (x n,w) t ni ) 2 i=1 "49
50 <latexit sha1_base64="mselxol8+csqoz9dzmua0i0z984=">aaab7hicbvbns8naej34wetx1aoxxslus0 w<latexit sha1_base64="gme7o Learning by Optimization E(w) "50
51 <latexit sha1_base64="mselxol8+csqoz9dzmua0i0z984=">aaab7hicbvbns8naej34wetx1aoxxslus0 w<latexit sha1_base64="gme7o Learning by Optimization Gradient Descent E(w) "51
52 <latexit sha1_base64="y1sga0hc1+vyvzud2bsehdxorhg=">aaab63icb <latexit sha1_base64="dhgzajuhkm6fz1n54zeef5aaur4=">aaab63icbvbns8n <latexit sha1_base64="dp <latexit sha1_base64="bf8kdamfcwhxbnbq2/som65km0i=">aaab63icbvbnswmxej2tx7v+vt16crahxsq <latexit sha1_base64="w7zcg9cyd8hi5p6sduv7uqvj5bg=">aaaci3icbvbds8mwfe3n9/ya+uhlcagtcbqikiiiiocjgvuatyzbln2csvusvbml/8ux/4ovpijiiw/+f9otoe4pbe7oufcm9/gxz0rb9odvmpqemz2bxygvli2vrfbw1psqsishdrlxslz9ujszkdy005y2y0lb+jy2/nvz3g/dualyfn7oyuw9af2qbyyanlk3cnzf0buoh0+wir7ew24fhadsbhji6sygnqool2r3o9n3na/nupwqxbdhwh+ju5aqkndvrby5vygkgoaacfcq49ix9tj8jue0k7ujojgqw+jtjqehckq8dlrjhren0snbjm0jnr6ppztseeonhw8qbeibmvry8t+vk+jgyetzgceahmt8ujbwrcocb4 Network Training Parameter Optimization by Gradient Descent Goal: Minimize E(w) cannot be solved analytically has many local minima Iterative optimization procedure w[0] let be the starting vector for the weights (initialization) w[t] is the weight vector in the t-th iteration gradient descent step: in iteration [t+1] update the weight vector: w[t + 1] - with learning rate or step size slide adapted from David Stutz "52
53 Network Training - Approaches slide taken from David Stutz "53
54 <latexit sha1_base64="w7zcg9cyd8hi5p6sduv7uqvj5bg=">aaaci3icbvbds8mwfe3n9/ya+uhlcagtcbqikiiiiocjgvuatyzbln2csvusvbml/8ux/4ovpijiiw/+f9otoe4pbe7oufcm9/gxz0rb9odvmpqemz2bxygvli2vrfbw1psqsishdrlxslz9ujszkdy005y2y0lb+jy2/nvz3g/dualyfn7oyuw9af2qbyyanlk3cnzf0buoh0+wir7ew24fhadsbhji6sygnqool2r3o9n3na/nupwqxbdhwh+ju5aqkndvrby5vygkgoaacfcq49ix9tj8jue0k7ujojgqw+jtjqehckq8dlrjhren0snbjm0jnr6ppztseeonhw8qbeibmvry8t+vk+jgyetzgceahmt8ujbwrcoc <latexit sha1_base64="1ddb+j8hnjc23veomsa2sh4ck2g=">aaab9hicb <latexit sha1_base64="ou7jv5ivmelq3e/mqatd1v9md0k=">aaacshicbzblswmxfiuz9v1fvzdugkwoigvgbn0oggguk1gvounwj83uajizkoxshvl5bly68ze4cagio9mhqnulgcn37r1jtprypo3rpjulsfgjyanpmfls3pzcymvp+vwnmsk0srkeqmsinovm0qzhhtplvfeqeacx0e1rz7+4o0qzrj6zbkodar3jykbawbrwwvuw2fscmgc3xvve4xsf3sew/sj4c/sdeakwhysguz+cmgw4pg4lrt1vfn9kzlqik1w37vyl/xxeuftrsbph5clvjyqtvbrcqeuw56ymyhvrcadf2c80tyhcqoe2rjqgqa7yfhafxrekjene2smn7tofezkirbsisp0czlue9xrwp6+vmxgvyjlmm0mlgvwuzxybbpdsxw2mkdg8awuqxexbmbkgm5wx2zdtcn7ol/+k8+2659a9053q4cewjmm0itzqdxlofx2ie9ratut <latexit sha1_base64="+kfq56y9+8ypkzn1gifexnnghyu=">aaacgxicbvdlssnafj3uv62vqks3g0vonyurqvdsemflbfuajibjdnkonuzczmrsqn7djb/ixouilnxl Backpropagation = Parameter Optimization by Gradient Descent General gradient descent step: w[t + 1] More specifically for weight w (l) ij w[t + 1] (l) ij = w[t](l) n (l) ij How to n (l) ij slide taken from David Stutz "54
55 <latexit sha1_base64="n6j4eskoxv3u/jrxk3h7gcpjxce=">aaacunicbvlps8mwgm3m7/lr6tflcajbwdgkobdf8ojrwelgnsxn0i0zauvyvamlf6mgxvxdvhhqs62tuflb4phe+74kl/eiwtvy1nuhode/sli0vfjaxvvf2cxvbd/qmfaunwgoqtx0igacb6wbharrroor6ql25z1cdps7r6y0d4mbsclwlqqbcj9taozyyzxx+x1afbumn2ifo4l5umxpv6sjelchtunr0bf00/6pld2ncma7sgtzhp/clpezvc1x+2nlpmitv6y6nsw8c+wcvfbev2751emenjysacqi1i3biqcdegwccpavnfizinah0mutawmimw6nw0gyvg+ydvzdzvyaemhodqreap1izzglgz6e1gbkf1orbv+knfigioefdlsrhwsmir7kiztcmqoimybqxc1zme0rrsiyvyizeozpk8+c28o6bdxt66pk+vkexzlarxuoimx0jm7rjbpcdut <latexit sha1_base64="+iikcj4/nkyieeacrqeqqj47njs=">aaact3iclvfbs8mwge3rbc7b1edfgkpyxkyray+gdetwcyk74dzlmqyulk1rkjpm6v/0wtf/jdmn6aaihwqo5zvfsxi+n2juksv6mmyl5zxvtcx6dmnza3snt7txl2esmknhkiwi6sjjgowkpqhipbkjggkxkybbuxr2gy9esbryozwiscdaj5z6fcolksf31vyfwkk7qkjrxoc1wwv9yjoj+k5cn9khpmckaqov4f/6v4doxxpswec393nt7xgm0ft5bwsnl7dk1qjgirania8mvxvy720vxhfaumimsdmyruh1kqe3zitntmnjior76jg0noqoilktjhjp4zfmpoihqh+u4ij9opggqmpb4gplgfrxzveg5e+9vqz8005cerqrwvh4ij9muivwuetouugwygmnebzuvxxiltjrkb3qra7bnv/yiqgfl2yrzn+w85xlsrwzcaaoqqhy4aruwa2oghrartm4n7dhmwemy/pmdyw1jcnmpvhw5vmn/fxzka==</latexit> <latexit sha1_base64="dwe0w8sr6q7ahp4hwop6rlczcju=">aaackhicbvdlssnafj34rpuvdelmsajtwpkiobu14mzlbfuanobjdnjoo3kwm1fiyoe48vfcicjsrv/ipa2orqcgzj3nxu7c44smcmkyy21hcwl5zbwwvlzf2nza1nd2mykiocynhlcatx0kckm+augqgwmhncdpya <latexit sha1_base64="xe6esrqbwrgdcp+boog7ktkbnr0=">aaacm3icbzdlssnafiyn9vbrrerszwar2ouleue3skeecvxbxqcpytkdtnnojmfmyikh7+tgf3ehiatf3poottqc2vrdwm93zuhm+d2qualm88xilcwula9kv3nr6xubw/ntnbomiofjdqcsee0xsciojzvffspnubdku4w03mffwm/ceyfpwg/vkcrth3u59shgsimnf217auhydpfqfdf46fdisjt8gket035yfxdzkunggbq7hcnk0ambi6efukorldj5glk2x4lzxpqaapiq Backpropagation = Parameter Optimization by Gradient Descent Remember: y (l) i = f z (l) i = f m (l 1) X j=0 w (l) ij y(l 1) j 1 A Using chain rule: dropping [t] for n (l) ij (l) (l) (l) ij := (l) (l) (l) ij Second (l) (l) ij (l 1) = y j n (l) ij = (l) (l 1) i y j "55
56 <latexit sha1_base64="vmmz4pzxtbmsbj/g+yunxvf98us=">aaacjnichvfds8mwfe3r19z8qpros3aig8jorvsu6uaeh3yy4d5gnsxn0i2ypivjlvn6c/xdvvlvzozgc1o8edicc25ucq4fmyqvbx8a5tlyyupabj1f2njc2rz2dpsysgqmdryxslr9jamjndquvyy0y0fq6dps8p9vrnrrhqhji/6ohjhphqjpauaxupryrhe3r5hchn1ks/dhtjmdf1ui3uagnloxeooibm89xnotz1piberpypvf+3dwdvnz9ln1jxgevbqr9rjginamoagmvfesd7cx4sqkxgggpow4dqy66ehuzeiwdxnjyosfuz90noqojlkbjupm4kfmejcihd5cwte725giumph6gtninrazmsj8jetk6jgvjtshiekcpw9kegyvbec7qb2qcbysaegcauq3wrxaomwln5gxofgzh95etspk45dcr5oirwrsrw5sa8oqak44azuwb2ogwbarsfwj <latexit sha1_base64="risbdnzg0uyft2tsppjxrggqt1u=">aaac0nicdvjda9swfjw9r877srfhvyiffydtwqqf9ivqwgz7gkydpi3ejpevorgvzspj61khjbhx/bq97sfsx0xodeub7ylgco6538oqzpsool+ef+v2nbv3tu4hdx4+evyks/30vjw1jhresl7k8wwrypmgi800p+evpljiod3llo4a/9knkhurxylevdqp8eywnbgshzv2fse5xmtefzaayq7fpik87nm/xcjlexo+f4v61skdibyunzdmnnfjlkbwdcyesaql1lahspojlsbczxn+tsvrenmdb6bojwbnhgszue5nbi1kop1pd8fwi9v6pjgodpakjsjppxv1o6xbtyba0awthaedn/g0jhvbhsyckzvguaut0/reolvbxctayxkbz3tsomafvylznstcl46zwryu7gknl+x6hmgfuosic8oc67m66wvif/ngtc73e8nevwsqykpqxnoos9jcf06zpetzhqoysoz6hwso3xg0+wwbwwk6ofimob30udrhh3e7b4ftorbac/achacbpxaa3ofjmale++dv3hfvq3/ix/nf/o8rqe+1mc/anfn//aemsdui</latexit> <latexit sha1_base64="fjnbw2rmlltusd5me5n3amt7ipu=">aaacqhicbzc7swnbemb34ju+opy2i0fiemkdcnoogo2fhyixqu5y7g3mkiv7d3bnhhjkt7pxt7cztrfqxnbktqwyhwmlh983w8z+glqkjbb9abwmpmdm5+yxiotlyyurpbx1k51kikodjzjr1whtieumdrqo4tpvwkjaqiponqzzxg0olzl4evspebhrxciunkgx/flddrxjuzsyhyjj2vdfk6+c7tjvwedbvz1w6senw26qratulrbiztkmrhkdllb9ut <latexit sha1_base64="lhermcp9fj2t27uwysvwaoutovg=">aaaconicbvbnaxsxfnsmazu6x05zzexufna0nsttac6g0fdiiyceajvgxs9awwsla7wl9dbpivz35zjfkvspvftqehlnd4js+jamhramm294epmuuhgigl/e2pp1p8+eb7xovhz1+s3b5ua7gcllzxif5tlxxwk1xarf+yba8uncc5olkg+t+d7ch55wbusufkbv8cijuyvswsg4kw4efy+vf9rgprymmjjlatutmq5nmcvw9eg93soh5cn4uirf2pohh0m79n/g6pnlfcyqwyuwas2mm2ipuzhutojosar+tmiktnakh3hzipzkrmy4aiapmsmsfbbzqkewyetgwbpeudanuz5yvngmm8gut6/xb6dmcjpr9xtgpxo/ywlmtjulbjkjmdmpvyx4p29uqrotwagkerhid4vsumli8ajhpbgam5cvi5rp4f6k2yy6csg13xalkicnpyadbocehxl0pbx7bvxhbtpg75g Backpropagation = Parameter Optimization by Gradient Descent First term for output layer: with (L+1) i sum-of-squared error n (L+1) i = (L+1) i E n (w) = 1 2 apple P i=1 y (L+1) i (x n,w) t ni = y i (x n,w) t (L+1) i (L+1) i CX (L+1) (L+1) i y (L+1) i (x n,w) t ni 2 = y (L+1) i (x n,w) t (L+1) (L+1) i = f 0 z (L+1) i "56
57 <latexit sha1_base64="uxnukpp993462mzygv6tprmfp94=">aaacghicbvfda9swfjw9ry77srfhvyifgco21bqdjkjgyqz22mhsfuleymp1olasjxs9khn/jv2vve3hdcandt3axracnxmpujo3k5v0gme/gvdw7tt37+3c7z14+ojxk/7u02nxvfbarbsqskczd6ckgqlkvhbawua6u3csnx9s9znvyj0szfdclzdtfglklgvht6x9h8kcfpjuzutidrt6mkzjbrmok5jblfzrt6mjlobnfff929zqmc3nswmlbpooydg6emli5xkfq5q4sqf12zg181q32ivwwi88j5vtnxzjng2jtd+ir/gm6e3aojagxr2l/z/johcvboncceemlc5xvrczcwvnl6kclfyc8yvmptrcg5vvmwab+tizc5ox1h+ddmp+7ai5dm6tm9+poa7cda0l/6dnk8zfz2ppygrbimuh8kprlgi7dbqqfgsqtqdcwolnpwlfff7od9bzibdrx74jjt+owdxix94ndj90ceyq5+qfiqgj++sqfczhzeie+r0mgtfbmzamo3avzjetydb5npf/kjz4ayhsv9c=</latexit> <latexit sha1_base64="elmf0crwayi7eutjtaasaymfpic=">aaab+hicbvbns8naej3ur1 <latexit sha1_base64="a/qtmvgdfbdzulf4fwrq88i3k7e=">aaacahicbzdlssnafizpvnz6i7pw4wawcbwhj <latexit sha1_base64="lrq/hp7kvqnkbjti4j8nxe5hmia=">aaab+nicbvbns8naen34wetxqkcvi0woccurqu9s8olbqwx7a Backpropagation = Parameter Optimization by Gradient Descent First term for all other layers: (l) i (l) i m X (l+1) = f 0 z (l) i this can be evaluated recursively starting with the output layer L+1: j=1 w (l+1) ji (l+1) j (L+1) i w (l+1) 1i w (l+1) m (l+1) i "57
58 <latexit sha1_base64="xe6esrqbwrgdcp+boog7ktkbnr0=">aaacm3icbzdlssnafiyn9vbrrerszwar2ouleue3skeecvxbxqcpytkdtnnojmfmyikh7+tgf3ehiatf3poottqc2vrdwm93zuhm+d2qualm88xilcwula9kv3nr6xubw/ntnbomiofjdqcsee0xsciojzvffspnubdku4w03mffwm/ceyfpwg/vkcrth3u59shgsimnf217auhydpfqfdf46fdisjt8gket035yfxdzkunggbq7hcnk0ambi6efukorldj5glk2x4lzxpqaapiq6u <latexit sha1_base64="ou7jv5ivmelq3e/mqatd1v9md0k=">aaacshicbzblswmxfiuz9v1fvzdugkwoigvgbn0oggguk1gvounwj83uajizkoxshvl5bly68ze4cagio9mhqnulgcn37r1jtprypo3rpjulsfgjyanpmfls3pzcymvp+vwnmsk0srkeqmsinovm0qzhhtplvfeqeacx0e1rz7+4o0qzrj6zbkodar3jykbawbrwwvuw2fscmgc3xvve4xsf3sew/sj4c/sdeakwhysguz+cmgw4pg4lrt1vfn9kzlqik1w37vyl/xxeuftrsbph5clvjyqtvbrcqeuw56ymyhvrcadf2c80tyhcqoe2rjqgqa7yfhafxrekjene2smn7tofezkirbsisp0czlue9xrwp6+vmxgvyjlmm0mlgvwuzxybbpdsxw2mkdg8awuqxexbmbkgm5wx2zdtcn7ol/+k8+2659a9053q4cewjmm0itzqdxlofx2ie9ratutq Backpropagation = Parameter Optimization by Gradient Descent determine the required derivatives using: use derivatives to update all weights in iteration [t+1] : w[t + 1] (l) n (l) ij = w[t](l) ij = (l) (l 1) i y n (l) ij "58
59 Gradient & Training Only two things need to be implemented to define new layer: forward pass error back propagation Watch overfitting loss/energy test validation train epochs "59
60 Gradient & Training Only two things need to be implemented to define new layer: forward pass error back propagation Watch overfitting loss/energy test validation train epochs "60
61 Gradient & Training Only two things need to be implemented to define new layer: forward pass error back propagation Watch overfitting loss/energy test validation train epochs "61
62 Overview Today Recent successes of Deep Learning necessarily incomplete What is Deep Learning? Neural network basics Supervised learning with error backpropagation [reading Bishop chapter ] Convolutional Neural Networks "62
63 Overview of Deep Learning "63
64 Multistage Hubel&Wiesel Architecture Slide: Y.LeCun [Hubel & Wiesel 1962] simple cells detect local features complex cells pool the outputs of simple cells within a retinotopic neighborhood. Cognitron / Neocognitron [Fukushima ] Also HMAX [Poggio ] Convolutional Networks [LeCun 1988-present]
65 Convolutional Neural Networks LeCun et al Neural network with specialized connectivity structure
66 Convnet Successes Handwritten text/digits MNIST (0.17% error [Ciresan et al. 2011]) Arabic & Chinese [Ciresan et al. 2012] Simpler recognition benchmarks CIFAR-10 (9.3% error [Wan et al. 2013]) Traffic sign recognition 0.56% error vs 1.16% for humans [Ciresan et al. 2011] But (until recently) less good at more complex datasets E.g. Caltech-101/256 (few training examples)
67 Characteristics of Convnets Feed-forward: Convolve input Non-linearity (rectified linear) Pooling (local max) / (=subsampling) Supervised Train convolutional filters by back-propagating classification error Feature maps Pooling Non-linearity Convolution (Learned) Input Image [LeCun et al. 1989]
68 68
69 69
70 70
71 71
72 72
73 73
74 74
75 Characteristics of Convnets Feed-forward: Convolve input Non-linearity (rectified linear) Pooling (local max) / (=subsampling) Supervised Train convolutional filters by back-propagating classification error Feature maps Pooling Non-linearity Convolution (Learned) Input Image [LeCun et al. 1989]
76 Application to ImageNet [Deng et al. CVPR 2009] used 1.2 million images of 1000 labeled classes (overall ~ 14 million labeled images, 20k classes) Images gathered from Internet Human labels via Amazon Turk [NIPS 2012]
77 Krizhevsky et al. [NIPS2012] Same model as LeCun 98 but: - Bigger model (8 layers) - More data (10 6 vs 10 3 images) - GPU implementation (50x speedup over CPU) - Better regularization (DropOut) 7 hidden layers, 650,000 neurons, 60,000,000 parameters Trained on 2 GPUs for a week
78 ImageNet Classification 2012 Krizhevsky et al % error (top-5) Next best (non-convnet) 26.2% error Top-5 error rate % SuperVision ISI Oxford INRIA Amsterdam
79 Commercial Deployment Google & Baidu, Spring 2013 for personal image search
Deep Neural Networks:
Deep Neural Networks: Part II Convolutional Neural Network (CNN) Yuan-Kai Wang, 2016 Web site of this course: http://pattern-recognition.weebly.com source: CNN for ImageClassification, by S. Lazebnik,
More informationCS 2750: Machine Learning. Neural Networks. Prof. Adriana Kovashka University of Pittsburgh April 13, 2016
CS 2750: Machine Learning Neural Networks Prof. Adriana Kovashka University of Pittsburgh April 13, 2016 Plan for today Neural network definition and examples Training neural networks (backprop) Convolutional
More informationConvolutional Neural Networks. Computer Vision Jia-Bin Huang, Virginia Tech
Convolutional Neural Networks Computer Vision Jia-Bin Huang, Virginia Tech Today s class Overview Convolutional Neural Network (CNN) Training CNN Understanding and Visualizing CNN Image Categorization:
More informationComputer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning for Object Categorization 14.01.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period
More informationDeep Learning for Computer Vision II
IIIT Hyderabad Deep Learning for Computer Vision II C. V. Jawahar Paradigm Shift Feature Extraction (SIFT, HoG, ) Part Models / Encoding Classifier Sparrow Feature Learning Classifier Sparrow L 1 L 2 L
More informationMachine Learning. Deep Learning. Eric Xing (and Pengtao Xie) , Fall Lecture 8, October 6, Eric CMU,
Machine Learning 10-701, Fall 2015 Deep Learning Eric Xing (and Pengtao Xie) Lecture 8, October 6, 2015 Eric Xing @ CMU, 2015 1 A perennial challenge in computer vision: feature engineering SIFT Spin image
More informationCS 1674: Intro to Computer Vision. Neural Networks. Prof. Adriana Kovashka University of Pittsburgh November 16, 2016
CS 1674: Intro to Computer Vision Neural Networks Prof. Adriana Kovashka University of Pittsburgh November 16, 2016 Announcements Please watch the videos I sent you, if you haven t yet (that s your reading)
More informationDeep Learning in Visual Recognition. Thanks Da Zhang for the slides
Deep Learning in Visual Recognition Thanks Da Zhang for the slides Deep Learning is Everywhere 2 Roadmap Introduction Convolutional Neural Network Application Image Classification Object Detection Object
More informationPart-Based Models for Object Class Recognition Part 3
High Level Computer Vision! Part-Based Models for Object Class Recognition Part 3 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de! http://www.d2.mpi-inf.mpg.de/cv ! State-of-the-Art
More informationImageNet Classification with Deep Convolutional Neural Networks
ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky Ilya Sutskever Geoffrey Hinton University of Toronto Canada Paper with same name to appear in NIPS 2012 Main idea Architecture
More informationINTRODUCTION TO DEEP LEARNING
INTRODUCTION TO DEEP LEARNING CONTENTS Introduction to deep learning Contents 1. Examples 2. Machine learning 3. Neural networks 4. Deep learning 5. Convolutional neural networks 6. Conclusion 7. Additional
More informationSpatial Localization and Detection. Lecture 8-1
Lecture 8: Spatial Localization and Detection Lecture 8-1 Administrative - Project Proposals were due on Saturday Homework 2 due Friday 2/5 Homework 1 grades out this week Midterm will be in-class on Wednesday
More informationProgress in Computer Vision in the Last Decade & Open Problems: People Detection & Human Pose Estimation
Progress in Computer Vision in the Last Decade & Open Problems: People Detection & Human Pose Estimation Bernt Schiele Max Planck Institute for Informatics & Saarland University, Saarland Informatics Campus
More informationMachine Learning 13. week
Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of
More informationCOMP9444 Neural Networks and Deep Learning 7. Image Processing. COMP9444 c Alan Blair, 2017
COMP9444 Neural Networks and Deep Learning 7. Image Processing COMP9444 17s2 Image Processing 1 Outline Image Datasets and Tasks Convolution in Detail AlexNet Weight Initialization Batch Normalization
More informationDeep Learning. Vladimir Golkov Technical University of Munich Computer Vision Group
Deep Learning Vladimir Golkov Technical University of Munich Computer Vision Group 1D Input, 1D Output target input 2 2D Input, 1D Output: Data Distribution Complexity Imagine many dimensions (data occupies
More informationPart-Based Models for Object Class Recognition Part 2
High Level Computer Vision Part-Based Models for Object Class Recognition Part 2 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de https://www.mpi-inf.mpg.de/hlcv Class of Object
More informationPart-Based Models for Object Class Recognition Part 2
High Level Computer Vision Part-Based Models for Object Class Recognition Part 2 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de https://www.mpi-inf.mpg.de/hlcv Class of Object
More informationLearning-based Methods in Vision
Learning-based Methods in Vision 16-824 Sparsity and Deep Learning Motivation Multitude of hand-designed features currently in use in vision - SIFT, HoG, LBP, MSER, etc. Even the best approaches, just
More informationDeep Learning. Deep Learning. Practical Application Automatically Adding Sounds To Silent Movies
http://blog.csdn.net/zouxy09/article/details/8775360 Automatic Colorization of Black and White Images Automatically Adding Sounds To Silent Movies Traditionally this was done by hand with human effort
More informationCOMP 551 Applied Machine Learning Lecture 16: Deep Learning
COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all
More informationLecture 17: Neural Networks and Deep Learning. Instructor: Saravanan Thirumuruganathan
Lecture 17: Neural Networks and Deep Learning Instructor: Saravanan Thirumuruganathan Outline Perceptron Neural Networks Deep Learning Convolutional Neural Networks Recurrent Neural Networks Auto Encoders
More informationIntroduction to Neural Networks
Introduction to Neural Networks Jakob Verbeek 2017-2018 Biological motivation Neuron is basic computational unit of the brain about 10^11 neurons in human brain Simplified neuron model as linear threshold
More informationDeep Learning with Tensorflow AlexNet
Machine Learning and Computer Vision Group Deep Learning with Tensorflow http://cvml.ist.ac.at/courses/dlwt_w17/ AlexNet Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton, "Imagenet classification
More informationDeep Learning. Visualizing and Understanding Convolutional Networks. Christopher Funk. Pennsylvania State University.
Visualizing and Understanding Convolutional Networks Christopher Pennsylvania State University February 23, 2015 Some Slide Information taken from Pierre Sermanet (Google) presentation on and Computer
More informationObject Recognition II
Object Recognition II Linda Shapiro EE/CSE 576 with CNN slides from Ross Girshick 1 Outline Object detection the task, evaluation, datasets Convolutional Neural Networks (CNNs) overview and history Region-based
More informationImage and Video Understanding
Image and Video Understanding 2VO 70.095 WS Christoph Feichtenhofer, Axel Pinz Slide credits: Many thanks to all the great computer vision researchers on which this presentation relies on. Most material
More informationSEMANTIC COMPUTING. Lecture 8: Introduction to Deep Learning. TU Dresden, 7 December Dagmar Gromann International Center For Computational Logic
SEMANTIC COMPUTING Lecture 8: Introduction to Deep Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 7 December 2018 Overview Introduction Deep Learning General Neural Networks
More informationNeural Networks and Deep Learning
Neural Networks and Deep Learning Example Learning Problem Example Learning Problem Celebrity Faces in the Wild Machine Learning Pipeline Raw data Feature extract. Feature computation Inference: prediction,
More informationDeep (1) Matthieu Cord LIP6 / UPMC Paris 6
Deep (1) Matthieu Cord LIP6 / UPMC Paris 6 Syllabus 1. Whole traditional (old) visual recognition pipeline 2. Introduction to Neural Nets 3. Deep Nets for image classification To do : Voir la leçon inaugurale
More informationComputer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning Applications 11.01.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period starts
More informationComputer Vision Lecture 16
Announcements Computer Vision Lecture 16 Deep Learning Applications 11.01.2017 Seminar registration period starts on Friday We will offer a lab course in the summer semester Deep Robot Learning Topic:
More informationDeep Learning Basic Lecture - Complex Systems & Artificial Intelligence 2017/18 (VO) Asan Agibetov, PhD.
Deep Learning 861.061 Basic Lecture - Complex Systems & Artificial Intelligence 2017/18 (VO) Asan Agibetov, PhD asan.agibetov@meduniwien.ac.at Medical University of Vienna Center for Medical Statistics,
More informationAdvanced Introduction to Machine Learning, CMU-10715
Advanced Introduction to Machine Learning, CMU-10715 Deep Learning Barnabás Póczos, Sept 17 Credits Many of the pictures, results, and other materials are taken from: Ruslan Salakhutdinov Joshua Bengio
More informationYiqi Yan. May 10, 2017
Yiqi Yan May 10, 2017 P a r t I F u n d a m e n t a l B a c k g r o u n d s Convolution Single Filter Multiple Filters 3 Convolution: case study, 2 filters 4 Convolution: receptive field receptive field
More informationMachine Learning. MGS Lecture 3: Deep Learning
Dr Michel F. Valstar http://cs.nott.ac.uk/~mfv/ Machine Learning MGS Lecture 3: Deep Learning Dr Michel F. Valstar http://cs.nott.ac.uk/~mfv/ WHAT IS DEEP LEARNING? Shallow network: Only one hidden layer
More informationArtificial Neural Networks. Introduction to Computational Neuroscience Ardi Tampuu
Artificial Neural Networks Introduction to Computational Neuroscience Ardi Tampuu 7.0.206 Artificial neural network NB! Inspired by biology, not based on biology! Applications Automatic speech recognition
More informationYOLO9000: Better, Faster, Stronger
YOLO9000: Better, Faster, Stronger Date: January 24, 2018 Prepared by Haris Khan (University of Toronto) Haris Khan CSC2548: Machine Learning in Computer Vision 1 Overview 1. Motivation for one-shot object
More informationDeep learning for dense per-pixel prediction. Chunhua Shen The University of Adelaide, Australia
Deep learning for dense per-pixel prediction Chunhua Shen The University of Adelaide, Australia Image understanding Classification error Convolution Neural Networks 0.3 0.2 0.1 Image Classification [Krizhevsky
More informationCSE 559A: Computer Vision
CSE 559A: Computer Vision Fall 2018: T-R: 11:30-1pm @ Lopata 101 Instructor: Ayan Chakrabarti (ayan@wustl.edu). Course Staff: Zhihao Xia, Charlie Wu, Han Liu http://www.cse.wustl.edu/~ayan/courses/cse559a/
More informationIntroduction to Neural Networks
Introduction to Neural Networks Machine Learning and Object Recognition 2016-2017 Course website: http://thoth.inrialpes.fr/~verbeek/mlor.16.17.php Biological motivation Neuron is basic computational unit
More informationStructured Prediction using Convolutional Neural Networks
Overview Structured Prediction using Convolutional Neural Networks Bohyung Han bhhan@postech.ac.kr Computer Vision Lab. Convolutional Neural Networks (CNNs) Structured predictions for low level computer
More informationKnow your data - many types of networks
Architectures Know your data - many types of networks Fixed length representation Variable length representation Online video sequences, or samples of different sizes Images Specific architectures for
More informationAll You Want To Know About CNNs. Yukun Zhu
All You Want To Know About CNNs Yukun Zhu Deep Learning Deep Learning Image from http://imgur.com/ Deep Learning Image from http://imgur.com/ Deep Learning Image from http://imgur.com/ Deep Learning Image
More informationLearning. Learning agents Inductive learning. Neural Networks. Different Learning Scenarios Evaluation
Learning Learning agents Inductive learning Different Learning Scenarios Evaluation Slides based on Slides by Russell/Norvig, Ronald Williams, and Torsten Reil Material from Russell & Norvig, chapters
More informationInception and Residual Networks. Hantao Zhang. Deep Learning with Python.
Inception and Residual Networks Hantao Zhang Deep Learning with Python https://en.wikipedia.org/wiki/residual_neural_network Deep Neural Network Progress from Large Scale Visual Recognition Challenge (ILSVRC)
More informationTutorial on Keras CAP ADVANCED COMPUTER VISION SPRING 2018 KISHAN S ATHREY
Tutorial on Keras CAP 6412 - ADVANCED COMPUTER VISION SPRING 2018 KISHAN S ATHREY Deep learning packages TensorFlow Google PyTorch Facebook AI research Keras Francois Chollet (now at Google) Chainer Company
More informationStudy of Residual Networks for Image Recognition
Study of Residual Networks for Image Recognition Mohammad Sadegh Ebrahimi Stanford University sadegh@stanford.edu Hossein Karkeh Abadi Stanford University hosseink@stanford.edu Abstract Deep neural networks
More informationA Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images Marc Aurelio Ranzato Yann LeCun Courant Institute of Mathematical Sciences New York University - New York, NY 10003 Abstract
More informationConvolutional Neural Networks + Neural Style Transfer. Justin Johnson 2/1/2017
Convolutional Neural Networks + Neural Style Transfer Justin Johnson 2/1/2017 Outline Convolutional Neural Networks Convolution Pooling Feature Visualization Neural Style Transfer Feature Inversion Texture
More informationA Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images Marc Aurelio Ranzato Yann LeCun Courant Institute of Mathematical Sciences New York University - New York, NY 10003 Abstract
More informationDL Tutorial. Xudong Cao
DL Tutorial Xudong Cao Historical Line 1960s Perceptron 1980s MLP BP algorithm 2006 RBM unsupervised learning 2012 AlexNet ImageNet Comp. 2014 GoogleNet VGGNet ImageNet Comp. Rule based AI algorithm Game
More informationSimplifying ConvNets for Fast Learning
Simplifying ConvNets for Fast Learning Franck Mamalet 1 and Christophe Garcia 2 1 Orange Labs, 4 rue du Clos Courtel, 35512 Cesson-Sévigné, France, franck.mamalet@orange.com 2 LIRIS, CNRS, Insa de Lyon,
More informationConvolutional Neural Networks
Lecturer: Barnabas Poczos Introduction to Machine Learning (Lecture Notes) Convolutional Neural Networks Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.
More informationInternational Journal of Computer Engineering and Applications, Volume XII, Special Issue, September 18,
REAL-TIME OBJECT DETECTION WITH CONVOLUTION NEURAL NETWORK USING KERAS Asmita Goswami [1], Lokesh Soni [2 ] Department of Information Technology [1] Jaipur Engineering College and Research Center Jaipur[2]
More informationWeakly Supervised Object Recognition with Convolutional Neural Networks
GDR-ISIS, Paris March 20, 2015 Weakly Supervised Object Recognition with Convolutional Neural Networks Ivan Laptev ivan.laptev@inria.fr WILLOW, INRIA/ENS/CNRS, Paris Joint work with: Maxime Oquab Leon
More informationMulti-Glance Attention Models For Image Classification
Multi-Glance Attention Models For Image Classification Chinmay Duvedi Stanford University Stanford, CA cduvedi@stanford.edu Pararth Shah Stanford University Stanford, CA pararth@stanford.edu Abstract We
More informationLearning Visual Semantics: Models, Massive Computation, and Innovative Applications
Learning Visual Semantics: Models, Massive Computation, and Innovative Applications Part II: Visual Features and Representations Liangliang Cao, IBM Watson Research Center Evolvement of Visual Features
More informationCMU Lecture 18: Deep learning and Vision: Convolutional neural networks. Teacher: Gianni A. Di Caro
CMU 15-781 Lecture 18: Deep learning and Vision: Convolutional neural networks Teacher: Gianni A. Di Caro DEEP, SHALLOW, CONNECTED, SPARSE? Fully connected multi-layer feed-forward perceptrons: More powerful
More informationConvolutional Neural Networks. CSC 4510/9010 Andrew Keenan
Convolutional Neural Networks CSC 4510/9010 Andrew Keenan Neural network to Convolutional Neural Network 0.4 0.7 0.3 Neural Network Basics 0.4 0.7 0.3 Individual Neuron Network Backpropagation 1. 2. 3.
More informationDeep Learning for Vision: Tricks of the Trade
Deep Learning for Vision: Tricks of the Trade Marc'Aurelio Ranzato Facebook, AI Group www.cs.toronto.edu/~ranzato BAVM Friday, 4 October 2013 Ideal Features Ideal Feature Extractor - window, right - chair,
More informationDEEP NEURAL NETWORKS FOR OBJECT DETECTION
DEEP NEURAL NETWORKS FOR OBJECT DETECTION Sergey Nikolenko Steklov Institute of Mathematics at St. Petersburg October 21, 2017, St. Petersburg, Russia Outline Bird s eye overview of deep learning Convolutional
More informationIntroduction to Deep Learning
ENEE698A : Machine Learning Seminar Introduction to Deep Learning Raviteja Vemulapalli Image credit: [LeCun 1998] Resources Unsupervised feature learning and deep learning (UFLDL) tutorial (http://ufldl.stanford.edu/wiki/index.php/ufldl_tutorial)
More informationDeep neural networks II
Deep neural networks II May 31 st, 2018 Yong Jae Lee UC Davis Many slides from Rob Fergus, Svetlana Lazebnik, Jia-Bin Huang, Derek Hoiem, Adriana Kovashka, Why (convolutional) neural networks? State of
More informationRyerson University CP8208. Soft Computing and Machine Intelligence. Naive Road-Detection using CNNS. Authors: Sarah Asiri - Domenic Curro
Ryerson University CP8208 Soft Computing and Machine Intelligence Naive Road-Detection using CNNS Authors: Sarah Asiri - Domenic Curro April 24 2016 Contents 1 Abstract 2 2 Introduction 2 3 Motivation
More informationarxiv: v1 [cs.cv] 6 Jul 2016
arxiv:607.079v [cs.cv] 6 Jul 206 Deep CORAL: Correlation Alignment for Deep Domain Adaptation Baochen Sun and Kate Saenko University of Massachusetts Lowell, Boston University Abstract. Deep neural networks
More informationDeep Learning Workshop. Nov. 20, 2015 Andrew Fishberg, Rowan Zellers
Deep Learning Workshop Nov. 20, 2015 Andrew Fishberg, Rowan Zellers Why deep learning? The ImageNet Challenge Goal: image classification with 1000 categories Top 5 error rate of 15%. Krizhevsky, Alex,
More informationLecture 20: Neural Networks for NLP. Zubin Pahuja
Lecture 20: Neural Networks for NLP Zubin Pahuja zpahuja2@illinois.edu courses.engr.illinois.edu/cs447 CS447: Natural Language Processing 1 Today s Lecture Feed-forward neural networks as classifiers simple
More informationDeep learning for object detection. Slides from Svetlana Lazebnik and many others
Deep learning for object detection Slides from Svetlana Lazebnik and many others Recent developments in object detection 80% PASCAL VOC mean0average0precision0(map) 70% 60% 50% 40% 30% 20% 10% Before deep
More informationLogical Rhythm - Class 3. August 27, 2018
Logical Rhythm - Class 3 August 27, 2018 In this Class Neural Networks (Intro To Deep Learning) Decision Trees Ensemble Methods(Random Forest) Hyperparameter Optimisation and Bias Variance Tradeoff Biological
More informationBidirectional Recurrent Convolutional Networks for Video Super-Resolution
Bidirectional Recurrent Convolutional Networks for Video Super-Resolution Qi Zhang & Yan Huang Center for Research on Intelligent Perception and Computing (CRIPAC) National Laboratory of Pattern Recognition
More informationEncoder-Decoder Networks for Semantic Segmentation. Sachin Mehta
Encoder-Decoder Networks for Semantic Segmentation Sachin Mehta Outline > Overview of Semantic Segmentation > Encoder-Decoder Networks > Results What is Semantic Segmentation? Input: RGB Image Output:
More informationVideo Object Segmentation using Deep Learning
Video Object Segmentation using Deep Learning Update Presentation, Week 3 Zack While Advised by: Rui Hou, Dr. Chen Chen, and Dr. Mubarak Shah June 2, 2017 Youngstown State University 1 Table of Contents
More informationNeural Networks. Single-layer neural network. CSE 446: Machine Learning Emily Fox University of Washington March 10, /10/2017
3/0/207 Neural Networks Emily Fox University of Washington March 0, 207 Slides adapted from Ali Farhadi (via Carlos Guestrin and Luke Zettlemoyer) Single-layer neural network 3/0/207 Perceptron as a neural
More informationIMPLEMENTING DEEP LEARNING USING CUDNN 이예하 VUNO INC.
IMPLEMENTING DEEP LEARNING USING CUDNN 이예하 VUNO INC. CONTENTS Deep Learning Review Implementation on GPU using cudnn Optimization Issues Introduction to VUNO-Net DEEP LEARNING REVIEW BRIEF HISTORY OF NEURAL
More informationDeep Learning & Neural Networks
Deep Learning & Neural Networks Machine Learning CSE4546 Sham Kakade University of Washington November 29, 2016 Sham Kakade 1 Announcements: HW4 posted Poster Session Thurs, Dec 8 Today: Review: EM Neural
More informationDynamic Routing Between Capsules
Report Explainable Machine Learning Dynamic Routing Between Capsules Author: Michael Dorkenwald Supervisor: Dr. Ullrich Köthe 28. Juni 2018 Inhaltsverzeichnis 1 Introduction 2 2 Motivation 2 3 CapusleNet
More informationDeep Learning & Feature Learning Methods for Vision
Deep Learning & Feature Learning Methods for Vision CVPR 2012 Tutorial: 9am-5:30pm Rob Fergus (NYU) Kai Yu (Baidu) Marc Aurelio Ranzato (Google) Honglak Lee (Michigan) Ruslan Salakhutdinov (U. Toronto)
More informationDeconvolutions in Convolutional Neural Networks
Overview Deconvolutions in Convolutional Neural Networks Bohyung Han bhhan@postech.ac.kr Computer Vision Lab. Convolutional Neural Networks (CNNs) Deconvolutions in CNNs Applications Network visualization
More informationCS 523: Multimedia Systems
CS 523: Multimedia Systems Angus Forbes creativecoding.evl.uic.edu/courses/cs523 Today - Convolutional Neural Networks - Work on Project 1 http://playground.tensorflow.org/ Convolutional Neural Networks
More informationCENG 783. Special topics in. Deep Learning. AlchemyAPI. Week 11. Sinan Kalkan
CENG 783 Special topics in Deep Learning AlchemyAPI Week 11 Sinan Kalkan TRAINING A CNN Fig: http://www.robots.ox.ac.uk/~vgg/practicals/cnn/ Feed-forward pass Note that this is written in terms of the
More informationNeural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer
More informationDeformable Part Models
CS 1674: Intro to Computer Vision Deformable Part Models Prof. Adriana Kovashka University of Pittsburgh November 9, 2016 Today: Object category detection Window-based approaches: Last time: Viola-Jones
More informationCharacter Recognition Using Convolutional Neural Networks
Character Recognition Using Convolutional Neural Networks David Bouchain Seminar Statistical Learning Theory University of Ulm, Germany Institute for Neural Information Processing Winter 2006/2007 Abstract
More informationDeep Learning. Volker Tresp Summer 2014
Deep Learning Volker Tresp Summer 2014 1 Neural Network Winter and Revival While Machine Learning was flourishing, there was a Neural Network winter (late 1990 s until late 2000 s) Around 2010 there
More informationWeighted Convolutional Neural Network. Ensemble.
Weighted Convolutional Neural Network Ensemble Xavier Frazão and Luís A. Alexandre Dept. of Informatics, Univ. Beira Interior and Instituto de Telecomunicações Covilhã, Portugal xavierfrazao@gmail.com
More informationSingle Image Depth Estimation via Deep Learning
Single Image Depth Estimation via Deep Learning Wei Song Stanford University Stanford, CA Abstract The goal of the project is to apply direct supervised deep learning to the problem of monocular depth
More informationHigh Level Computer Vision
High Level Computer Vision Part-Based Models for Object Class Recognition Part 2 Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de http://www.d2.mpi-inf.mpg.de/cv Please Note No
More informationFlow-Based Video Recognition
Flow-Based Video Recognition Jifeng Dai Visual Computing Group, Microsoft Research Asia Joint work with Xizhou Zhu*, Yuwen Xiong*, Yujie Wang*, Lu Yuan and Yichen Wei (* interns) Talk pipeline Introduction
More informationGlobal Optimality in Neural Network Training
Global Optimality in Neural Network Training Benjamin D. Haeffele and René Vidal Johns Hopkins University, Center for Imaging Science. Baltimore, USA Questions in Deep Learning Architecture Design Optimization
More informationLSTM and its variants for visual recognition. Xiaodan Liang Sun Yat-sen University
LSTM and its variants for visual recognition Xiaodan Liang xdliang328@gmail.com Sun Yat-sen University Outline Context Modelling with CNN LSTM and its Variants LSTM Architecture Variants Application in
More informationNeural Network Neurons
Neural Networks Neural Network Neurons 1 Receives n inputs (plus a bias term) Multiplies each input by its weight Applies activation function to the sum of results Outputs result Activation Functions Given
More informationCategory Recognition. Jia-Bin Huang Virginia Tech ECE 6554 Advanced Computer Vision
Category Recognition Jia-Bin Huang Virginia Tech ECE 6554 Advanced Computer Vision Administrative stuffs Presentation and discussion leads assigned https://docs.google.com/spreadsheets/d/1p5pfycio5flq
More informationCost-alleviative Learning for Deep Convolutional Neural Network-based Facial Part Labeling
[DOI: 10.2197/ipsjtcva.7.99] Express Paper Cost-alleviative Learning for Deep Convolutional Neural Network-based Facial Part Labeling Takayoshi Yamashita 1,a) Takaya Nakamura 1 Hiroshi Fukui 1,b) Yuji
More informationDEEP LEARNING REVIEW. Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature Presented by Divya Chitimalla
DEEP LEARNING REVIEW Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature 2015 -Presented by Divya Chitimalla What is deep learning Deep learning allows computational models that are composed of multiple
More informationMachine Learning With Python. Bin Chen Nov. 7, 2017 Research Computing Center
Machine Learning With Python Bin Chen Nov. 7, 2017 Research Computing Center Outline Introduction to Machine Learning (ML) Introduction to Neural Network (NN) Introduction to Deep Learning NN Introduction
More informationComo funciona o Deep Learning
Como funciona o Deep Learning Moacir Ponti (com ajuda de Gabriel Paranhos da Costa) ICMC, Universidade de São Paulo Contact: www.icmc.usp.br/~moacir moacir@icmc.usp.br Uberlandia-MG/Brazil October, 2017
More informationSupervised Hashing for Image Retrieval via Image Representation Learning
Supervised Hashing for Image Retrieval via Image Representation Learning Rongkai Xia, Yan Pan, Cong Liu (Sun Yat-Sen University) Hanjiang Lai, Shuicheng Yan (National University of Singapore) Finding Similar
More informationMachine Learning. The Breadth of ML Neural Networks & Deep Learning. Marc Toussaint. Duy Nguyen-Tuong. University of Stuttgart
Machine Learning The Breadth of ML Neural Networks & Deep Learning Marc Toussaint University of Stuttgart Duy Nguyen-Tuong Bosch Center for Artificial Intelligence Summer 2017 Neural Networks Consider
More informationMocha.jl. Deep Learning in Julia. Chiyuan Zhang CSAIL, MIT
Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning with multi-layer (3~30) neural networks, on a huge training set. State-of-the-art on many AI tasks Computer Vision:
More information