Perception Systems for Autonomous Vehicles using Energy-Efficient Deep Neural Networks
|
|
|
- Toby Bishop
- 5 years ago
- Views:
Transcription
1 Perception Systems for Autonomous Vehicles using Energy-Efficient Deep Neural Networks Forrest Iandola, Ben Landen, Kyle Bertin, Kurt Keutzer and the DeepScale Team
2 OFFLINE MAPS CAMERA SENSORS ULTRASONIC PATH PLANNING & ACTUATION RADAR LIDAR REAL-TIME PERCEPTION THE FLOW IMPLEMENTING AUTONOMOUS DRIVING
3 What does a car need to see? What does a car need to see? Note: above visuals are an artist s rendering created to help convey concepts. They should not be judged for accuracy.
4 What does a car need to see? Object Detection Cyclist (99%) Cyclist (99%) Pedestrian (99%) Pedestrian (99%) Vehicle (99%) Vehicle (98%) Vehicle (100%) Vehicle (100%) Vehicle (99%) Note: above visuals are an artist s rendering created to help convey concepts. They should not be judged for accuracy.
5 What does a car need to see? Distance Vehicle (99%) 15m Vehicle (100%) 10m Vehicle (98%) 20m Vehicle (100%) 14m Vehicle (99%) 18m Cyclist (99%) 16m Cyclist (99%) 14m Pedestrian (99%) 7m Pedestrian (99%) 7m Note: above visuals are an artist s rendering created to help convey concepts. They should not be judged for accuracy.
6 What does a car need to see? Object Tracking Vehicle (99%) 15m ID: 5 (95 frames) Vehicle (100%) 10m ID: 1 (135 frames) Vehicle (98%) 20m ID: 4 (140 frames) Vehicle (100%) 14m ID: 2 (140 frames) Vehicle (99%) 18m ID: 3 (140 frames) Cyclist (99%) 16m ID: 6 (90 frames) Cyclist (99%) 14m ID: 7 (95 frames) Pedestrian (99%) 7m ID: 8 (60 frames) Pedestrian (99%) 7m ID: 9 (60 frames) Note: above visuals are an artist s rendering created to help convey concepts. They should not be judged for accuracy.
7 What does a car need to see? Free Space & Driveable Area Vehicle (99%) 15m ID: 5 (95 frames) Vehicle (100%) 10m ID: 1 (135 frames) Vehicle (98%) 20m ID: 4 (140 frames) Vehicle (100%) 14m ID: 2 (140 frames) Vehicle (99%) 18m ID: 3 (140 frames) Cyclist (99%) 16m ID: 6 (90 frames) Cyclist (99%) 14m ID: 7 (95 frames) Pedestrian (99%) 7m ID: 8 (60 frames) Pedestrian (99%) 7m ID: 9 (60 frames) Note: above visuals are an artist s rendering created to help convey concepts. They should not be judged for accuracy.
8 What does a car need to see? Lane Recognition Vehicle (99%) 15m ID: 5 (95 frames) Vehicle (100%) 10m ID: 1 (135 frames) Vehicle (98%) 20m ID: 4 (140 frames) Vehicle (100%) 14m ID: 2 (140 frames) Vehicle (99%) 18m ID: 3 (140 frames) Cyclist (99%) 16m ID: 6 (90 frames) Cyclist (99%) 14m ID: 7 (95 frames) Pedestrian (99%) 7m ID: 8 (60 frames) Pedestrian (99%) 7m ID: 9 (60 frames) Note: above visuals are an artist s rendering created to help convey concepts. They should not be judged for accuracy.
9 Today's autonomous cars require a lot of computing hardware! and perception is the most computationally-intensive part of the software stack Audi BMW + Intel Ford cted-cars-long-road-autonomousvehicles
10 Big computers = expensive cars
11 As a workaround, companies want people to share autonomous vehicles to amortize hardware costs
12 As a workaround, companies want people to share autonomous vehicles to amortize hardware costs
13 Shared autonomous vehicles will likely have some of the downsides as public transportation
14 Will better computer chips make autonomous cars affordable?
15 Will better computer chips make autonomous cars affordable?
16 Deep Learning Processors have arrived! THE SERVER SIDE Platform Computation (GFLOPS/s) NVIDIA K [1] (32-bit float) NVIDIA V100 [2] (16-bit float) Memory Bandwidth (GB/s) 208 (GDDR5) 900 (HBM2) Computationto-bandwidth ratio Power (TDP Watts) Year (yikes!) Uh-oh Processors are improving much faster than Memory. [1] [2] (PCIe version)
17 Deep Learning Processors have arrived! MOBILE PLATFORMS Device Cores Computation (GFLOPS/s) Samsung Galaxy Note 3 Huawei P20 NVIDIA Jetson Xavier [3,4] Arm Mali 120 T-628 GPU [1] (32-bit float) Kirin NPU [2] (16-bit float) NVIDIA Tensor Cores (8 32 int) Memory Bandwidth (GB/s) 12.8 (LPDDR3) 30 (LPDDR4X) Computationto-bandwidth ratio System Power (TDP Watts) 9.3 ~ (ouch!) (yikes!) ~ to 30 (multiple modes) Year 2018 [1] [2] [3] [4]
18 What will the next generation Deep Learning servers look like?
19 What will the next generation Deep Learning servers look like? 20 TOP/W COMPUTATION Platform Efficiency (TOP/s/W) Computation (TOP/s) NVIDIA K20 [1] (32-bit float) NVIDIA V100 [2] (16-bit float) Memory Bandwidth (TB/s) (GDDR5) (HBM2) Next-gen: * TOP/W (HBM3) [3] Computationto-bandwidth ratio Power (TDP Watts) Year (oh no!) (est.) [1] [2] (PCIe version) [3] * Assuming half the power is spent on computation, and the other half is spent on memory and other devices. 20 TOP/s/W * 20W * 0.5 = 2500 TOP/s
20 Small Neural Nets to the rescue
21 squeeze (verb): to make an AI system use less resources using whatever means necessary
22 Memory Footprint and Bandwidth Computational Operations Power and Energy Time squeeze (verb): to make an AI system use less resources using whatever means necessary
23 Memory Footprint and Bandwidth Computational Operations Power and Energy Time squeeze (verb): to make an AI system use less resources using whatever means necessary New DNN Models Applicationspecific Quantization and Pruning Superior Implementations Differentiated Data and Training Strategies
24 Most CV Applications Rely on Only a Few Core CV Capabilities Image Classification Object Detection Semantic Segmentation And the best accuracy for each of these capabilities is given by Convolutional Neural Nets
![But We Need a Very Different Kind of DNN VGG16[1]](/docs-images/95/123533450/images/25-0.jpg "model: - Parameter size: 552 MB - Memory: 93")
![Watts 2-4 GB IOT Devices 100's MHz <1Watt <1 GB [1]](/docs-images/95/123533450/images/25-4.jpg "K. Simonyan and A. Zisserman.")
25 But We Need a Very Different Kind of DNN VGG16[1] model: - Parameter size: 552 MB - Memory: 93 MB/image - Computation: 15.8 GFLOPs/image TitanX: 11 TFLOPS, 223 Watts, 12 GB Memory DGX-1, 170 TFLOPS, 3.2 KWatts, 128 GB Memory Smartphones 100's GFLOPs 3 Watts 2-4 GB IOT Devices 100's MHz <1Watt <1 GB [1] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arxiv: , 2014.
26 Speed more Related to Memory Accesses than Operations Samsung Exynos M1 Access Times Galaxy S7 L1 Cache/TLB L2 Cache L1 D-Cache (per core) L2 Cache (shared) Off-chip DRAM Size 32 KB 2 MB 4 GB Read Latency 4 cycles 22 cycles ~200 cycles Read Bandwidth 20.8 GB/s GB/s 28.7 GB/s
27 Energy More Related to Memory Accesses than operations (45nm 0.9V) 8b INT Mult Energy (pj) b FP Mult 32b FP Mult 5.5x 18.5x 64b Cache Read (32KB) 100x 64b Cache Read (1MB) 500x DRAM 10,000x Mark Horowitz, Computing s Energy Problem (and what we can do about it), ISSCC 2014
![10,000 DNN Architectural Configurations Later: SqueezeNet (2016) AlexNet [1] SqueezeNet [2] CNN Top-5 Accuracy ImageNet Model](/docs-images/95/123533450/images/28-0.jpg "Parameters Model Size AlexNet[1] 80.3% 60M 243MB SqueezeNet[2] 80.3% 1.2M 4.")
![8MB compresses to 500KB [1] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton.](/docs-images/95/123533450/images/28-2.jpg "\"Imagenet classification with deep convolutional neural networks.\" NIPS2012 [2] Iandola, Forrest N., et al.")
28 10,000 DNN Architectural Configurations Later: SqueezeNet (2016) AlexNet [1] SqueezeNet [2] CNN Top-5 Accuracy ImageNet Model Parameters Model Size AlexNet[1] 80.3% 60M 243MB SqueezeNet[2] 80.3% 1.2M 4.8MB compresses to 500KB [1] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." NIPS2012 [2] Iandola, Forrest N., et al. "SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 1MB model size." arxiv: (2016). (February 2016)
29 SqueezeNet: Immediate Success in Embedded Vision Apple CoreML NXP Embedded Vision Summit Qualcomm Facebook F8 Enabled embedded processor vendors (ARM, NXP, Qualcomm) to demo CNNs Quickly ported to all the major Deep Learning Frameworks
30 SqueezeDet for Object Detection (2017) Conv Det Filter feature map Input image ~2M model parameters 57 FPS 1.4 Joules Frame Bounding boxes Final detections Best Paper Award: Bichen Wu, Forrest Iandola, Peter H. Jin, and Kurt Keutzer SqueezeDet: Unified, small, low power fully convolutional neural networks for realtime object detection for autonomous driving. In Proceedings, CVPR Embedded Computer Vision Workshop, July 2017.
![SqueezeSeg: Semantic Segmentation for LIDAR (2018) LIDAR point cloud segmentation SqueezeSegV2: Higher accuracy: v1[1]: 64.6% -> v2[2]: 73.2% (+8.](/docs-images/95/123533450/images/31-0.jpg "6%) Better Sim2Real performance: v1[1]: 30% -> v2[2]: 57.4% (+27.4%) Outperforms v1 trained on real data w/o intensity [1] Wu, Bichen, et al.")
31 SqueezeSeg: Semantic Segmentation for LIDAR (2018) LIDAR point cloud segmentation SqueezeSegV2: Higher accuracy: v1[1]: 64.6% -> v2[2]: 73.2% (+8.6%) Better Sim2Real performance: v1[1]: 30% -> v2[2]: 57.4% (+27.4%) Outperforms v1 trained on real data w/o intensity [1] Wu, Bichen, et al. "Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud." ICRA18 [2] Wu, Bichen, et al. "SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud." arxiv: (2018).
32 Squeeze Family SqueezeNet DiracDeltaNet SqueezeNext SqueezeDet SqueezeSeg-{v1, v2} DNASNet ShiftNet Image Classification Object Detection Semantic Segmentation
33 Andrew Howard's MobileNets: Efficient On-Device Computer Vision Models Designed for efficiency on mobile phones. Family of pareto optimal models to target needs of the user. V1 based on Depthwise Separable Convolutions. V2 introduces Inverted Residuals and Linear Bottlenecks. Supports Classification, Detection, Segmentation and more. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications MobileNet V2: Inverted Residuals and Linear Bottlenecks
34 Model Compression DNN Architecture Search 10X 50X Slide Credit: Prof. Warren Gross (McGill Univ.)
35 Anatomy of a convolution layer x3 conv
36 Filters: Kernel Reduction x3 conv x reduction in model parameters 1
37 Filters/Channel Reduction x3 conv x3 conv x reduction in model parameters 128
38 Model Distillation/Compression Model Distillation Li, et al. Mimicking Very Efficient Network for Object Detection. CVPR, 2017.
39 Examples of what's on a DNN Architect's Palette Spatial Convolution e.g. 3x3 Pointwise Convolution 1x1 Depthwise Convolution Channel Shuffle Shift
40 The Art of Small Model Design Small Neural Nets Are Beautiful ESWeek 2017 The palette of an adept mobile/embedded DNN designer has grown very rich! Overall architecture: economize on layers while retaining accuracy Layer types Kernel reduction: 5 x 5 3 x 3 1 x 1 Channel reduction: e.g. FireLayer Experiment with novel layer types that consume no FLOPS Shuffle Shift Model distillation: let big models teach smaller ones Apply pruning Tailor bit precision (aka quantization) to target processor Iandola, Forrest, and Kurt Keutzer. "Small neural nets are beautiful: enabling embedded systems with small deep-neural-network architectures." In Proceedings of the Twelfth International Conference on Hardware/Software Codesign and System Synthesis Companion, p. 1. ACM, (ESWeek 2017). Also, (arxiv: )
41 Artistic/Engineering Process of Designing a Deep Neural Net Manual design: Each iteration to evaluate a point in the design space is very expensive Exploration limited by human imagination
42 Can we automate this? Manual design: Each iteration to evaluate a point in the design space is very expensive Exploration limited by human imagination
43 DNAS: Differentiable Neural Architecture Search Differentiable Neural Architecture Search: Extremely fast: 8 GPUs, 24 hours Can search for different conditions case-by-case Optimize for actual latency Bichen Wu, Kurt Keutzer, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia
44 DNAS in context (FLOPs to normalize comparison) ImageNet top-1 Accuracy -- Good DNASNet: (ours) Acc: 74.2%, FLOPs: 295M Search Cost: 216 GPU-hrs MnasNet: [6] Acc: 74.0, FLOPs: 317M Search Cost*: 91,000 GPU-hrs MobileNetV2: [4] Acc: 71.8%, FLOPs: 300M AMC: [5] Acc: 70.8%, FLOPs: 150M * Estimated from the paper description PNAS: [2] Acc: 74.2%, FLOPs: 588M Search cost*: 6,000 GPU-hrs NAS: [1] Acc: 74.0%, FLOPs: 564M Search cost: 48,000 GPU-hrs DARTS: [3] Acc: 73.1%, FLOPs: 595M Search cost: 288 GPU-hrs X-axis: FLOPs Y-axis: accuracy Mark size: search cost Circles: search cost unknown More FLOPs - BAD [1] Zoph, Barret, et al. "Learning transferable architectures for scalable image recognition." arxiv preprint arxiv: (2017). [2] Liu, Chenxi, et al. "Progressive neural architecture search." arxiv preprint arxiv: (2017). [3] Liu, Hanxiao, Karen Simonyan, and Yiming Yang. "Darts: Differentiable architecture search." arxiv preprint arxiv: (2018) [4] Sandler, Mark, et al. "MobileNetV2: Inverted Residuals and Linear Bottlenecks. CVPR18 [5] He, Yihui, et al. "Amc: Automl for model compression and acceleration on mobile devices." Proceedings of the European Conference on Computer Vision (ECCV) [6] Tan, Mingxing, et al. "Mnasnet: Platform-aware neural architecture search for mobile." arxiv preprint arxiv: (2018).
45 DNAS for device-aware search For different targeted devices, both DNASNets achieve similar accuracy. However, per target DNN optimization was required NET Latency on iphonex Latency on Samsung S8 DNAS-iPhoneX ms ms (20% slower) DNAS-S ms (25% slower) Top-1 acc 73.20% ms 73.27%
46 The Future: Breaking down the wall between DNN Design & Hardware Design DNN Designers Unaware of arithmetic intensity Floating point vs fixed point costs Memory hierarchy and latency NN HW Accelerator architects Using outdated models: - AlexNet - VGG16 Using irrelevant datasets: - MNIST - CIFAR
47 Key Takeaways Autonomous vehicles currently need thousands (or even hundreds of thousands) of dollars of computing hardware Processing is on a trajectory of rapid improvement (in operations-per-watt) but other aspects of the system (e.g. memory) are improving much more slowly today's neural networks will be choked by slow memory on tomorrow's DNN accelerators (this is already happening and will get worse) Designing new (smaller) neural networks helps with all of the following making full use of next-generation computing platforms reducing the hardware costs in autonomous vehicles enabling lower-cost, larger-scale rollouts of autonomous vehicles
Progressive Neural Architecture Search
Progressive Neural Architecture Search Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy 09/10/2018 @ECCV 1 Outline Introduction
Architecting new deep neural networks for embedded applications
Architecting new deep neural networks for embedded applications Forrest Iandola 1 Machine Learning in 2012 Sentiment Analysis LDA Object Detection Deformable Parts Model Word Prediction Linear Interpolation
Research Faculty Summit Systems Fueling future disruptions
Research Faculty Summit 2018 Systems Fueling future disruptions Efficient Edge Computing for Deep Neural Networks and Beyond Vivienne Sze In collaboration with Yu-Hsin Chen, Joel Emer, Tien-Ju Yang, Sertac
Real-time convolutional networks for sonar image classification in low-power embedded systems
Real-time convolutional networks for sonar image classification in low-power embedded systems Matias Valdenegro-Toro Ocean Systems Laboratory - School of Engineering & Physical Sciences Heriot-Watt University,
How to Estimate the Energy Consumption of Deep Neural Networks
How to Estimate the Energy Consumption of Deep Neural Networks Tien-Ju Yang, Yu-Hsin Chen, Joel Emer, Vivienne Sze MIT 1 Problem of DNNs Recognition Smart Drone AI Computation DNN 15k 300k OP/Px DPM 0.1k
DEEP NEURAL NETWORKS CHANGING THE AUTONOMOUS VEHICLE LANDSCAPE. Dennis Lui August 2017
DEEP NEURAL NETWORKS CHANGING THE AUTONOMOUS VEHICLE LANDSCAPE Dennis Lui August 2017 THE RISE OF GPU COMPUTING APPLICATIONS 10 7 10 6 GPU-Computing perf 1.5X per year 1000X by 2025 ALGORITHMS 10 5 1.1X
HENet: A Highly Efficient Convolutional Neural. Networks Optimized for Accuracy, Speed and Storage
HENet: A Highly Efficient Convolutional Neural Networks Optimized for Accuracy, Speed and Storage Qiuyu Zhu Shanghai University zhuqiuyu@staff.shu.edu.cn Ruixin Zhang Shanghai University chriszhang96@shu.edu.cn
Deep Learning with Tensorflow AlexNet
Machine Learning and Computer Vision Group Deep Learning with Tensorflow http://cvml.ist.ac.at/courses/dlwt_w17/ AlexNet Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton, "Imagenet classification
Object Detection and Its Implementation on Android Devices
Object Detection and Its Implementation on Android Devices Zhongjie Li Stanford University 450 Serra Mall, Stanford, CA 94305 jay2015@stanford.edu Rao Zhang Stanford University 450 Serra Mall, Stanford,
Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAs
Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAs Ritchie Zhao 1, Weinan Song 2, Wentao Zhang 2, Tianwei Xing 3, Jeng-Hau Lin 4, Mani Srivastava 3, Rajesh Gupta 4, Zhiru
Small is the New Big: Data Analytics on the Edge
Small is the New Big: Data Analytics on the Edge An overview of processors and algorithms for deep learning techniques on the edge Dr. Abhay Samant VP Engineering, Hiller Measurements Adjunct Faculty,
Switched by Input: Power Efficient Structure for RRAMbased Convolutional Neural Network
Switched by Input: Power Efficient Structure for RRAMbased Convolutional Neural Network Lixue Xia, Tianqi Tang, Wenqin Huangfu, Ming Cheng, Xiling Yin, Boxun Li, Yu Wang, Huazhong Yang Dept. of E.E., Tsinghua
Deep Learning in Visual Recognition. Thanks Da Zhang for the slides
Deep Learning in Visual Recognition Thanks Da Zhang for the slides Deep Learning is Everywhere 2 Roadmap Introduction Convolutional Neural Network Application Image Classification Object Detection Object
Channel Locality Block: A Variant of Squeeze-and-Excitation
Channel Locality Block: A Variant of Squeeze-and-Excitation 1 st Huayu Li Northern Arizona University Flagstaff, United State Northern Arizona University hl459@nau.edu arxiv:1901.01493v1 [cs.lg] 6 Jan
Characterization and Benchmarking of Deep Learning. Natalia Vassilieva, PhD Sr. Research Manager
Characterization and Benchmarking of Deep Learning Natalia Vassilieva, PhD Sr. Research Manager Deep learning applications Vision Speech Text Other Search & information extraction Security/Video surveillance
REGION AVERAGE POOLING FOR CONTEXT-AWARE OBJECT DETECTION
REGION AVERAGE POOLING FOR CONTEXT-AWARE OBJECT DETECTION Kingsley Kuan 1, Gaurav Manek 1, Jie Lin 1, Yuan Fang 1, Vijay Chandrasekhar 1,2 Institute for Infocomm Research, A*STAR, Singapore 1 Nanyang Technological
Deep Learning Requirements for Autonomous Vehicles
Deep Learning Requirements for Autonomous Vehicles Pierre Paulin, Director of R&D Synopsys Inc. Chipex, 1 May 2018 1 Agenda Deep Learning and Convolutional Neural Networks for Embedded Vision Automotive
Deep Learning on Arm Cortex-M Microcontrollers. Rod Crawford Director Software Technologies, Arm
Deep Learning on Arm Cortex-M Microcontrollers Rod Crawford Director Software Technologies, Arm What is Machine Learning (ML)? Artificial Intelligence Machine Learning Deep Learning Neural Networks Additional
Implementation of Deep Convolutional Neural Net on a Digital Signal Processor
Implementation of Deep Convolutional Neural Net on a Digital Signal Processor Elaina Chai December 12, 2014 1. Abstract In this paper I will discuss the feasibility of an implementation of an algorithm
Learning Deep Representations for Visual Recognition
Learning Deep Representations for Visual Recognition CVPR 2018 Tutorial Kaiming He Facebook AI Research (FAIR) Deep Learning is Representation Learning Representation Learning: worth a conference name
Deep learning for object detection. Slides from Svetlana Lazebnik and many others
Deep learning for object detection Slides from Svetlana Lazebnik and many others Recent developments in object detection 80% PASCAL VOC mean0average0precision0(map) 70% 60% 50% 40% 30% 20% 10% Before deep
Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks
Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks Naveen Suda, Vikas Chandra *, Ganesh Dasika *, Abinash Mohanty, Yufei Ma, Sarma Vrudhula, Jae-sun Seo, Yu
Bandwidth-Centric Deep Learning Processing through Software-Hardware Co-Design
Bandwidth-Centric Deep Learning Processing through Software-Hardware Co-Design Song Yao 姚颂 Founder & CEO DeePhi Tech 深鉴科技 song.yao@deephi.tech Outline - About DeePhi Tech - Background - Bandwidth Matters
A NEW COMPUTING ERA JENSEN HUANG, FOUNDER & CEO GTC CHINA 2017
A NEW COMPUTING ERA JENSEN HUANG, FOUNDER & CEO GTC CHINA 2017 TWO FORCES DRIVING THE FUTURE OF COMPUTING 10 7 Transistors (thousands) 10 6 10 5 1.1X per year 10 4 10 3 10 2 1.5X per year Single-threaded
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations, and Hardware Implications
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations, and Hardware Implications Jongsoo Park Facebook AI System SW/HW Co-design Team Sep-21 2018 Team Introduction
Autonomous Driving Solutions
Autonomous Driving Solutions Oct, 2017 DrivePX2 & DriveWorks Marcus Oh (moh@nvidia.com) Sr. Solution Architect, NVIDIA This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA
arxiv: v1 [cs.ne] 23 Mar 2018
![arxiv: v1 [cs.ne] 23 Mar 2018](/thumbs/81/84223479.jpg "arxiv: v1 [cs.ne] 23 Mar 2018")
SqueezeNext: Hardware-Aware Neural Network Design arxiv:1803.10615v1 [cs.ne] 23 Mar 2018 Amir Gholami, Kiseok Kwon, Bichen Wu, Zizheng Tai, Xiangyu Yue, Peter Jin, Sicheng Zhao, Kurt Keutzer EECS, UC Berkeley
Deep Learning Accelerators
Deep Learning Accelerators Abhishek Srivastava (as29) Samarth Kulshreshtha (samarth5) University of Illinois, Urbana-Champaign Submitted as a requirement for CS 433 graduate student project Outline Introduction
Synscapes A photorealistic syntehtic dataset for street scene parsing Jonas Unger Department of Science and Technology Linköpings Universitet.
Synscapes A photorealistic syntehtic dataset for street scene parsing Jonas Unger Department of Science and Technology Linköpings Universitet 7D Labs VINNOVA https://7dlabs.com Photo-realistic image synthesis
Binary Convolutional Neural Network on RRAM
Binary Convolutional Neural Network on RRAM Tianqi Tang, Lixue Xia, Boxun Li, Yu Wang, Huazhong Yang Dept. of E.E, Tsinghua National Laboratory for Information Science and Technology (TNList) Tsinghua
DL Tutorial. Xudong Cao
DL Tutorial Xudong Cao Historical Line 1960s Perceptron 1980s MLP BP algorithm 2006 RBM unsupervised learning 2012 AlexNet ImageNet Comp. 2014 GoogleNet VGGNet ImageNet Comp. Rule based AI algorithm Game
[Supplementary Material] Improving Occlusion and Hard Negative Handling for Single-Stage Pedestrian Detectors
![[Supplementary Material] Improving Occlusion and Hard Negative Handling for Single-Stage Pedestrian Detectors](/thumbs/89/97941029.jpg "[Supplementary Material] Improving Occlusion and Hard Negative Handling for Single-Stage Pedestrian Detectors")
[Supplementary Material] Improving Occlusion and Hard Negative Handling for Single-Stage Pedestrian Detectors Junhyug Noh Soochan Lee Beomsu Kim Gunhee Kim Department of Computer Science and Engineering
Spatial Localization and Detection. Lecture 8-1
Lecture 8: Spatial Localization and Detection Lecture 8-1 Administrative - Project Proposals were due on Saturday Homework 2 due Friday 2/5 Homework 1 grades out this week Midterm will be in-class on Wednesday
Model Compression. Girish Varma IIIT Hyderabad
Model Compression Girish Varma IIIT Hyderabad http://bit.ly/2tpy1wu Big Huge Neural Network! AlexNet - 60 Million Parameters = 240 MB & the Humble Mobile Phone 1 GB RAM 1/2 Billion FLOPs NOT SO BAD! But
Revolutionizing the Datacenter
Power-Efficient Machine Learning using FPGAs on POWER Systems Ralph Wittig, Distinguished Engineer Office of the CTO, Xilinx Revolutionizing the Datacenter Join the Conversation #OpenPOWERSummit Top-5
THE NVIDIA DEEP LEARNING ACCELERATOR
THE NVIDIA DEEP LEARNING ACCELERATOR INTRODUCTION NVDLA NVIDIA Deep Learning Accelerator Developed as part of Xavier NVIDIA s SOC for autonomous driving applications Optimized for Convolutional Neural
Convolutional Neural Networks. Computer Vision Jia-Bin Huang, Virginia Tech
Convolutional Neural Networks Computer Vision Jia-Bin Huang, Virginia Tech Today s class Overview Convolutional Neural Network (CNN) Training CNN Understanding and Visualizing CNN Image Categorization:
Towards a Uniform Template-based Architecture for Accelerating 2D and 3D CNNs on FPGA
Towards a Uniform Template-based Architecture for Accelerating 2D and 3D CNNs on FPGA Junzhong Shen, You Huang, Zelong Wang, Yuran Qiao, Mei Wen, Chunyuan Zhang National University of Defense Technology,
How to Build Optimized ML Applications with Arm Software
How to Build Optimized ML Applications with Arm Software Arm Technical Symposia 2018 ML Group Overview Today we will talk about applied machine learning (ML) on Arm. My aim for today is to show you just
Layer-wise Performance Bottleneck Analysis of Deep Neural Networks
Layer-wise Performance Bottleneck Analysis of Deep Neural Networks Hengyu Zhao, Colin Weinshenker*, Mohamed Ibrahim*, Adwait Jog*, Jishen Zhao University of California, Santa Cruz, *The College of William
Object Detection. CS698N Final Project Presentation AKSHAT AGARWAL SIDDHARTH TANWAR
Object Detection CS698N Final Project Presentation AKSHAT AGARWAL SIDDHARTH TANWAR Problem Description Arguably the most important part of perception Long term goals for object recognition: Generalization
A Method to Estimate the Energy Consumption of Deep Neural Networks
A Method to Estimate the Consumption of Deep Neural Networks Tien-Ju Yang, Yu-Hsin Chen, Joel Emer, Vivienne Sze Massachusetts Institute of Technology, Cambridge, MA, USA {tjy, yhchen, jsemer, sze}@mit.edu
Proceedings of the International MultiConference of Engineers and Computer Scientists 2018 Vol I IMECS 2018, March 14-16, 2018, Hong Kong
, March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong TABLE I CLASSIFICATION ACCURACY OF DIFFERENT PRE-TRAINED MODELS ON THE TEST DATA
Fuzzy Set Theory in Computer Vision: Example 3
Fuzzy Set Theory in Computer Vision: Example 3 Derek T. Anderson and James M. Keller FUZZ-IEEE, July 2017 Overview Purpose of these slides are to make you aware of a few of the different CNN architectures
arxiv: v3 [cs.cv] 21 Jul 2017
![arxiv: v3 [cs.cv] 21 Jul 2017](/thumbs/89/98108508.jpg "arxiv: v3 [cs.cv] 21 Jul 2017")
Structural Compression of Convolutional Neural Networks Based on Greedy Filter Pruning Reza Abbasi-Asl Department of Electrical Engineering and Computer Sciences University of California, Berkeley abbasi@berkeley.edu
The Mont-Blanc approach towards Exascale
http://www.montblanc-project.eu The Mont-Blanc approach towards Exascale Alex Ramirez Barcelona Supercomputing Center Disclaimer: Not only I speak for myself... All references to unavailable products are
Content-Based Image Recovery
Content-Based Image Recovery Hong-Yu Zhou and Jianxin Wu National Key Laboratory for Novel Software Technology Nanjing University, China zhouhy@lamda.nju.edu.cn wujx2001@nju.edu.cn Abstract. We propose
YOLO9000: Better, Faster, Stronger
YOLO9000: Better, Faster, Stronger Date: January 24, 2018 Prepared by Haris Khan (University of Toronto) Haris Khan CSC2548: Machine Learning in Computer Vision 1 Overview 1. Motivation for one-shot object
Addressing the Memory Wall
Lecture 26: Addressing the Memory Wall Parallel Computer Architecture and Programming CMU 15-418/15-618, Spring 2015 Tunes Cage the Elephant Back Against the Wall (Cage the Elephant) This song is for the
arxiv: v5 [cs.cv] 11 Dec 2018
![arxiv: v5 [cs.cv] 11 Dec 2018](/thumbs/93/111261947.jpg "arxiv: v5 [cs.cv] 11 Dec 2018")
Fire SSD: Wide Fire Modules based Single Shot Detector on Edge Device HengFui, Liau, 1, Nimmagadda, Yamini, 2 and YengLiong, Wong 1 {heng.hui.liau,yamini.nimmagadda,yeng.liong.wong}@intel.com, arxiv:1806.05363v5
DeepIndex for Accurate and Efficient Image Retrieval
DeepIndex for Accurate and Efficient Image Retrieval Yu Liu, Yanming Guo, Song Wu, Michael S. Lew Media Lab, Leiden Institute of Advance Computer Science Outline Motivation Proposed Approach Results Conclusions
NVIDIA PLATFORM FOR AI
NVIDIA PLATFORM FOR AI João Paulo Navarro, Solutions Architect - Linkedin i am ai HTTPS://WWW.YOUTUBE.COM/WATCH?V=GIZ7KYRWZGQ 2 NVIDIA Gaming VR AI & HPC Self-Driving Cars GPU Computing 3 GPU COMPUTING
Direct Multi-Scale Dual-Stream Network for Pedestrian Detection Sang-Il Jung and Ki-Sang Hong Image Information Processing Lab.
[ICIP 2017] Direct Multi-Scale Dual-Stream Network for Pedestrian Detection Sang-Il Jung and Ki-Sang Hong Image Information Processing Lab., POSTECH Pedestrian Detection Goal To draw bounding boxes that
CNNS FROM THE BASICS TO RECENT ADVANCES. Dmytro Mishkin Center for Machine Perception Czech Technical University in Prague
CNNS FROM THE BASICS TO RECENT ADVANCES Dmytro Mishkin Center for Machine Perception Czech Technical University in Prague ducha.aiki@gmail.com OUTLINE Short review of the CNN design Architecture progress
MobileFaceNets: Efficient CNNs for Accurate Real- Time Face Verification on Mobile Devices
MobileFaceNets: Efficient CNNs for Accurate Real- Time Face Verification on Mobile Devices Sheng Chen 1,2, Yang Liu 2, Xiang Gao 2, and Zhen Han 1 1 School of Computer and Information Technology, Beijing
Industrial Technology Research Institute, Hsinchu, Taiwan, R.O.C ǂ
Stop Line Detection and Distance Measurement for Road Intersection based on Deep Learning Neural Network Guan-Ting Lin 1, Patrisia Sherryl Santoso *1, Che-Tsung Lin *ǂ, Chia-Chi Tsai and Jiun-In Guo National
Making Sense of Artificial Intelligence: A Practical Guide
Making Sense of Artificial Intelligence: A Practical Guide JEDEC Mobile & IOT Forum Copyright 2018 Young Paik, Samsung Senior Director Product Planning Disclaimer This presentation and/or accompanying
GPU-Accelerated Deep Learning
GPU-Accelerated Deep Learning July 6 th, 2016. Greg Heinrich. Credits: Alison B. Lowndes, Julie Bernauer, Leo K. Tam. PRACTICAL DEEP LEARNING EXAMPLES Image Classification, Object Detection, Localization,
arxiv: v1 [cs.cv] 11 Feb 2018
![arxiv: v1 [cs.cv] 11 Feb 2018](/thumbs/79/79222664.jpg "arxiv: v1 [cs.cv] 11 Feb 2018")
arxiv:8.8v [cs.cv] Feb 8 - Partitioning of Deep Neural Networks with Feature Space Encoding for Resource-Constrained Internet-of-Things Platforms ABSTRACT Jong Hwan Ko, Taesik Na, Mohammad Faisal Amir,
Extend the shallow part of Single Shot MultiBox Detector via Convolutional Neural Network
Extend the shallow part of Single Shot MultiBox Detector via Convolutional Neural Network Liwen Zheng, Canmiao Fu, Yong Zhao * School of Electronic and Computer Engineering, Shenzhen Graduate School of
Inference Optimization Using TensorRT with Use Cases. Jack Han / 한재근 Solutions Architect NVIDIA
Inference Optimization Using TensorRT with Use Cases Jack Han / 한재근 Solutions Architect NVIDIA Search Image NLP Maps TensorRT 4 Adoption Use Cases Speech Video AI Inference is exploding 1 Billion Videos
Can FPGAs beat GPUs in accelerating next-generation Deep Neural Networks? Discussion of the FPGA 17 paper by Intel Corp. (Nurvitadhi et al.
Can FPGAs beat GPUs in accelerating next-generation Deep Neural Networks? Discussion of the FPGA 17 paper by Intel Corp. (Nurvitadhi et al.) Andreas Kurth 2017-12-05 1 In short: The situation Image credit:
Unified Deep Learning with CPU, GPU, and FPGA Technologies
Unified Deep Learning with CPU, GPU, and FPGA Technologies Allen Rush 1, Ashish Sirasao 2, Mike Ignatowski 1 1: Advanced Micro Devices, Inc., 2: Xilinx, Inc. Abstract Deep learning and complex machine
Fast Hardware For AI
Fast Hardware For AI Karl Freund karl@moorinsightsstrategy.com Sr. Analyst, AI and HPC Moor Insights & Strategy Follow my blogs covering Machine Learning Hardware on Forbes: http://www.forbes.com/sites/moorinsights
Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps
1 Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps Abhishek Sehgal and Nasser Kehtarnavaz Abstract Deep learning solutions are being increasingly used in
Unified, real-time object detection
Unified, real-time object detection Final Project Report, Group 02, 8 Nov 2016 Akshat Agarwal (13068), Siddharth Tanwar (13699) CS698N: Recent Advances in Computer Vision, Jul Nov 2016 Instructor: Gaurav
Building the Most Efficient Machine Learning System
Building the Most Efficient Machine Learning System Mellanox The Artificial Intelligence Interconnect Company June 2017 Mellanox Overview Company Headquarters Yokneam, Israel Sunnyvale, California Worldwide
Convolutional Neural Network Layer Reordering for Acceleration
R1-15 SASIMI 2016 Proceedings Convolutional Neural Network Layer Reordering for Acceleration Vijay Daultani Subhajit Chaudhury Kazuhisa Ishizaka System Platform Labs Value Co-creation Center System Platform
CNN optimization. Rassadin A
CNN optimization Rassadin A. 01.2017-02.2017 What to optimize? Training stage time consumption (CPU / GPU) Inference stage time consumption (CPU / GPU) Training stage memory consumption Inference stage
Maximizing Server Efficiency from μarch to ML accelerators. Michael Ferdman
Maximizing Server Efficiency from μarch to ML accelerators Michael Ferdman Maximizing Server Efficiency from μarch to ML accelerators Michael Ferdman Maximizing Server Efficiency with ML accelerators Michael
Accelerating your Embedded Vision / Machine Learning design with the revision Stack. Giles Peckham, Xilinx
Accelerating your Embedded Vision / Machine Learning design with the revision Stack Giles Peckham, Xilinx Xilinx Foundation at the Edge Vision Customers Using Xilinx >80 ADAS Models From 23 Makers >80
Building the Most Efficient Machine Learning System
Building the Most Efficient Machine Learning System Mellanox The Artificial Intelligence Interconnect Company June 2017 Mellanox Overview Company Headquarters Yokneam, Israel Sunnyvale, California Worldwide
CSCI 402: Computer Architectures. Parallel Processors (2) Fengguang Song Department of Computer & Information Science IUPUI.
CSCI 402: Computer Architectures Parallel Processors (2) Fengguang Song Department of Computer & Information Science IUPUI 6.6 - End Today s Contents GPU Cluster and its network topology The Roofline performance
Mocha.jl. Deep Learning in Julia. Chiyuan Zhang CSAIL, MIT
Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning with multi-layer (3~30) neural networks, on a huge training set. State-of-the-art on many AI tasks Computer Vision:
Object Detection Based on Deep Learning
Object Detection Based on Deep Learning Yurii Pashchenko AI Ukraine 2016, Kharkiv, 2016 Image classification (mostly what you ve seen) http://tutorial.caffe.berkeleyvision.org/caffe-cvpr15-detection.pdf
Computer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning for Object Categorization 14.01.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period
D-Pruner: Filter-Based Pruning Method for Deep Convolutional Neural Network
D-Pruner: Filter-Based Pruning Method for Deep Convolutional Neural Network Loc N. Huynh, Youngki Lee, Rajesh Krishna Balan Singapore Management University {nlhuynh.2014, youngkilee, rajesh}@smu.edu.sg
Flow-Based Video Recognition
Flow-Based Video Recognition Jifeng Dai Visual Computing Group, Microsoft Research Asia Joint work with Xizhou Zhu*, Yuwen Xiong*, Yujie Wang*, Lu Yuan and Yichen Wei (* interns) Talk pipeline Introduction
How to Build Optimized ML Applications with Arm Software
How to Build Optimized ML Applications with Arm Software Arm Technical Symposia 2018 Arm K.K. Senior FAE Ryuji Tanaka Overview Today we will talk about applied machine learning (ML) on Arm. My aim for
Scalable and Modularized RTL Compilation of Convolutional Neural Networks onto FPGA
Scalable and Modularized RTL Compilation of Convolutional Neural Networks onto FPGA Yufei Ma, Naveen Suda, Yu Cao, Jae-sun Seo, Sarma Vrudhula School of Electrical, Computer and Energy Engineering School
Object detection with CNNs
Object detection with CNNs 80% PASCAL VOC mean0average0precision0(map) 70% 60% 50% 40% 30% 20% 10% Before CNNs After CNNs 0% 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 year Region proposals
arxiv: v2 [cs.cv] 22 Dec 2016
![arxiv: v2 [cs.cv] 22 Dec 2016](/thumbs/72/66496604.jpg "arxiv: v2 [cs.cv] 22 Dec 2016")
SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving Bichen Wu 1, Forrest Iandola 1,2, Peter H. Jin 1, Kurt Keutzer 1,2 1 UC Berkeley,
The Power of Speech: Supporting Voice- Driven Commands in Small, Low-Power. Microcontrollers
Borrowing from an approach used for computer vision, we created a compact keyword spotting algorithm that supports voice-driven commands in edge devices that use a very small, low-power microcontroller.
Supplementary material for Analyzing Filters Toward Efficient ConvNet
Supplementary material for Analyzing Filters Toward Efficient Net Takumi Kobayashi National Institute of Advanced Industrial Science and Technology, Japan takumi.kobayashi@aist.go.jp A. Orthonormal Steerable
SDA: Software-Defined Accelerator for Large- Scale DNN Systems
SDA: Software-Defined Accelerator for Large- Scale DNN Systems Jian Ouyang, 1 Shiding Lin, 1 Wei Qi, Yong Wang, Bo Yu, Song Jiang, 2 1 Baidu, Inc. 2 Wayne State University Introduction of Baidu A dominant
Learning Semantic Video Captioning using Data Generated with Grand Theft Auto
A dark car is turning left on an exit Learning Semantic Video Captioning using Data Generated with Grand Theft Auto Alex Polis Polichroniadis Data Scientist, MSc Kolia Sadeghi Applied Mathematician, PhD
Study of Residual Networks for Image Recognition
Study of Residual Networks for Image Recognition Mohammad Sadegh Ebrahimi Stanford University sadegh@stanford.edu Hossein Karkeh Abadi Stanford University hosseink@stanford.edu Abstract Deep neural networks
Convolutional Neural Networks
Lecturer: Barnabas Poczos Introduction to Machine Learning (Lecture Notes) Convolutional Neural Networks Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.
Inception and Residual Networks. Hantao Zhang. Deep Learning with Python.
Inception and Residual Networks Hantao Zhang Deep Learning with Python https://en.wikipedia.org/wiki/residual_neural_network Deep Neural Network Progress from Large Scale Visual Recognition Challenge (ILSVRC)
MCMOT: Multi-Class Multi-Object Tracking using Changing Point Detection
MCMOT: Multi-Class Multi-Object Tracking using Changing Point Detection ILSVRC 2016 Object Detection from Video Byungjae Lee¹, Songguo Jin¹, Enkhbayar Erdenee¹, Mi Young Nam², Young Gui Jung², Phill Kyu
Efficient Segmentation-Aided Text Detection For Intelligent Robots
Efficient Segmentation-Aided Text Detection For Intelligent Robots Junting Zhang, Yuewei Na, Siyang Li, C.-C. Jay Kuo University of Southern California Outline Problem Definition and Motivation Related
direct hardware mapping of cnns on fpga-based smart cameras
direct hardware mapping of cnns on fpga-based smart cameras Workshop on Architecture of Smart Cameras Kamel ABDELOUAHAB, Francois BERRY, Maxime PELCAT, Jocelyn SEROT, Jean-Charles QUINTON Cordoba, June
A FRAMEWORK OF EXTRACTING MULTI-SCALE FEATURES USING MULTIPLE CONVOLUTIONAL NEURAL NETWORKS. Kuan-Chuan Peng and Tsuhan Chen
A FRAMEWORK OF EXTRACTING MULTI-SCALE FEATURES USING MULTIPLE CONVOLUTIONAL NEURAL NETWORKS Kuan-Chuan Peng and Tsuhan Chen School of Electrical and Computer Engineering, Cornell University, Ithaca, NY
Fei-Fei Li & Justin Johnson & Serena Yeung
Lecture 9-1 Administrative A2 due Wed May 2 Midterm: In-class Tue May 8. Covers material through Lecture 10 (Thu May 3). Sample midterm released on piazza. Midterm review session: Fri May 4 discussion
Arm s First-Generation Machine Learning Processor
Arm s First-Generation Machine Learning Processor Ian Bratt 2018 Arm Limited Introducing the Arm Machine Learning (ML) Processor Optimized ground-up architecture for machine learning processing Massive
Nvidia Jetson TX2 and its Software Toolset. João Fernandes 2017/2018
Nvidia Jetson TX2 and its Software Toolset João Fernandes 2017/2018 In this presentation Nvidia Jetson TX2: Hardware Nvidia Jetson TX2: Software Machine Learning: Neural Networks Convolutional Neural Networks
arxiv: v1 [cs.cv] 20 Dec 2016
![arxiv: v1 [cs.cv] 20 Dec 2016](/thumbs/73/68905842.jpg "arxiv: v1 [cs.cv] 20 Dec 2016")
End-to-End Pedestrian Collision Warning System based on a Convolutional Neural Network with Semantic Segmentation arxiv:1612.06558v1 [cs.cv] 20 Dec 2016 Heechul Jung heechul@dgist.ac.kr Min-Kook Choi mkchoi@dgist.ac.kr
ENDURING DIFFERENTIATION Timothy Lanfear
ENDURING DIFFERENTIATION Timothy Lanfear WHERE ARE WE? 2 LIFE AFTER DENNARD SCALING GPU-ACCELERATED PERFORMANCE 10 7 40 Years of Microprocessor Trend Data 10 6 10 5 10 4 10 3 10 2 Single-threaded perf
ENDURING DIFFERENTIATION. Timothy Lanfear
ENDURING DIFFERENTIATION Timothy Lanfear WHERE ARE WE? 2 LIFE AFTER DENNARD SCALING 10 7 40 Years of Microprocessor Trend Data 10 6 10 5 10 4 Transistors (thousands) 1.1X per year 10 3 10 2 Single-threaded
The Explosion in Neural Network Hardware
1 The Explosion in Neural Network Hardware Arm Summit, Cambridge, September 17 th, 2018 Trevor Mudge Bredt Family Professor of Computer Science and Engineering The, Ann Arbor 1 What Just Happened? 2 For