In Live Computer Vision
|
|
|
- April Lang
- 5 years ago
- Views:
Transcription
1 EVA 2 : Exploiting Temporal Redundancy In Live Computer Vision Mark Buckler, Philip Bedoukian, Suren Jayasuriya, Adrian Sampson International Symposium on Computer Architecture (ISCA) Tuesday June 5, 2018
2 Convolutional Neural Networks (s) 2
3 Convolutional Neural Networks (s) 3
4 FPGA Research Embedded Vision Accelerators ASIC Research Suda et al. Zhang et al. ShiDianNao Eyeriss Qiu et al. Farabet et al. EIE S Many more Industry Adoption Many more 4
5 Temporal Redundancy Frame 0 Frame 1 Frame 2 Frame 3 Input Change High Low Low Low 5
6 Temporal Redundancy Frame 0 Frame 1 Frame 2 Frame 3 Input Change High Low Low Low Cost to Process High High High High 6
7 Temporal Redundancy Frame 0 Frame 1 Frame 2 Frame 3 Input Change High Low Low Low Cost to Process High High Low High Low High Low 7
8 Talk Overview Background Algorithm Hardware Evaluation Conclusion 8
9 Talk Overview Background Algorithm Hardware Evaluation Conclusion 9
10 Common Structure in s Image Classification Object Detection Semantic Segmentation Image Captioning 10
11 Common Structure in s Intermediate Activations Frame 0 Prefix High energy Suffix Low energy Frame 1 Prefix High energy Suffix Low energy #MakeRyanGoslingTheNewLenna 11
12 Common Structure in s Intermediate Activations Key Frame Motion Prefix High energy Motion Suffix Low energy Predicted Frame Prefix High energy Suffix Low energy #MakeRyanGoslingTheNewLenna 12
13 Common Structure in s Intermediate Activations Key Frame Motion Prefix High energy Motion Suffix Low energy Predicted Frame Prefix Suffix Low energy #MakeRyanGoslingTheNewLenna 13
14 Talk Overview Background Algorithm Hardware Evaluation Conclusion 14
15 Activation Motion Compensation (AMC) Time Input Frame Vision Computation Vision Result Stored Activations Key Frame t Prefix Suffix Predicted Frame t+k Motion Estimation Motion Compensation Suffix Motion Vector Field Predicted Activations 15
16 Activation Motion Compensation (AMC) Time Input Frame Vision Computation Vision Result Stored Activations Key Frame t Prefix Suffix ~10 11 MACs Predicted Frame t+k Motion Estimation Motion Compensation Suffix ~10 7 Adds Motion Vector Field Predicted Activations 16
17 AMC Design Decisions How to perform motion estimation? How to perform motion compensation? Which frames are key frames? 17
18 AMC Design Decisions How to perform motion estimation? How to perform motion compensation? Which frames are key frames? 18
19 AMC Design Decisions How to perform motion estimation? How to perform motion compensation? Which frames are key frames? 19
20 AMC Design Decisions How to perform motion estimation? How to perform motion compensation? Which frames are key frames?? 20
21 AMC Design Decisions How to perform motion estimation? How to perform motion compensation? Which frames are key frames? 21
22 Motion Estimation We need to estimate the motion of activations by using pixels Prefix Suffix Motion Estimation Performed on Pixels Motion Compensation Performed on Activations Suffix 22
23 Pixels to Activations Input Image 3x3 Conv 64 Intermediate Activations 3x3 Conv 64 Intermediate Activations 23
24 Pixels to Activations: Receptive Fields C=3 C=64 C=64 w=h=8 Input Image 3x3 Conv 64 Intermediate Activations 3x3 Conv 64 Intermediate Activations 24
25 Pixels to Activations: Receptive Fields C=3 C=64 C=64 w=h=8 5x5 Receptive Field Input Image 3x3 Conv 64 Intermediate Activations 3x3 Conv 64 Intermediate Activations Estimate motion of activations by estimating motion of receptive fields 25
26 Receptive Field Block Motion Estimation (RFBME) Key Frame Predicted Frame 26
27 Receptive Field Block Motion Estimation (RFBME) Key Frame Predicted Frame 27
28 Receptive Field Block Motion Estimation (RFBME) Key Frame Predicted Frame 28
29 AMC Design Decisions How to perform motion estimation? How to perform motion compensation? Which frames are key frames? 29
30 Motion Compensation C=64 C=64 Vector: X = 2.5 Y = 2.5 Stored Activations Predicted Activations Subtract the vector to index into the stored activations Interpolate when necessary 30
31 AMC Design Decisions How to perform motion estimation? How to perform motion compensation? Which frames are key frames?? 31
32 When to Compute Key Frame? System needs a new key frame when motion estimation fails: De-occlusion New objects Rotation/scaling Lighting changes 32
33 When to Compute Key Frame? System needs a new key frame when motion estimation fails: De-occlusion New objects Rotation/scaling Lighting changes So, compute key frame when RFBME error exceeds set threshold Yes Prefix Input Frame Motion Estimation Error > Thresh? Suffix No Key Frame Motion Compensation Vision Result 33
34 Talk Overview Background Algorithm Hardware Evaluation Conclusion 34
35 Embedded Vision Accelerator Global Buffer Eyeriss (Conv) EIE (Full Connect) Prefix Suffix Y.-H. Chen, T. Krishna, J. S. Emer, and V. Sze, Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks, S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally, EIE: Efficient inference engine on compressed deep neural network, 35
36 Embedded Vision Accelerator Accelerator (EVA 2 ) Global Buffer EVA 2 Eyeriss (Conv) EIE (Full Connect) Motion Estimation Motion Compensation Prefix Suffix Y.-H. Chen, T. Krishna, J. S. Emer, and V. Sze, Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks, S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally, EIE: Efficient inference engine on compressed deep neural network, 36
37 Embedded Vision Accelerator Accelerator (EVA 2 ) Frame 0 37
38 Embedded Vision Accelerator Accelerator (EVA 2 ) Frame 0: Key frame 38
39 Embedded Vision Accelerator Accelerator (EVA 2 ) Frame 1 Motion Estimation 39
40 Embedded Vision Accelerator Accelerator (EVA 2 ) Frame 1: Predicted frame Motion Estimation Motion Compensation EVA 2 leverages sparse techniques to save 80-87% storage and computation 40
41 Talk Overview Background Algorithm Hardware Evaluation Conclusion 41
42 Evaluation Details Train/Validation Datasets Evaluated Networks Hardware Baseline EVA 2 Implementation YouTube Bounding Box: Object Detection & Classification AlexNet, Faster R- with VGGM and VGG16 Eyeriss & EIE performance scaled from papers Written in RTL, synthesized with 65nm TSMC 42
43 EVA 2 Area Overhead Total 65nm area: 74mm 2 EVA 2 takes up only 3.3% 43
44 Normalized Energy EVA 2 Energy Savings orig orig orig AlexNet Faster16 FasterM Eyeriss EIE EVA^2 Input Frame Prefix Suffix Vision Result 44
45 Normalized Energy EVA 2 Energy Savings Input Frame Key Frame orig pred orig pred orig pred AlexNet Faster16 FasterM Eyeriss EIE EVA^2 Motion Estimation Motion Compensation Suffix Vision Result 45
46 Normalized Energy EVA 2 Energy Savings Input Frame Key Frame orig pred avg orig pred avg orig pred avg AlexNet Faster16 FasterM Yes Prefix Motion Estimation Error > Thresh? Suffix No Motion Compensation Eyeriss EIE EVA^2 Vision Result 46
47 High Level EVA 2 Results Network Vision Task Keyframe % Accuracy Degredation Average Latency Savings AlexNet Classification 11% 0.8% top % 87.5% Faster R- VGG16 Detection 36% 0.7% map 61.7% 61.9% Faster R- VGGM Detection 37% 0.6% map 54.1% 54.7% Average Energy Savings EVA 2 enables 54-87% savings while incurring <1% accuracy degradation Adaptive key frame choice metric can be adjusted 47
48 Talk Overview Background Algorithm Hardware Evaluation Conclusion 48
49 Conclusion Temporal redundancy is an entirely new dimension for optimization AMC & EVA 2 improve efficiency and are highly general Applicable to many different applications (classification, detection, segmentation, etc) Hardware architectures (CPU, GPU, ASIC, etc) Motion estimation/compensation algorithms 49
50 EVA 2 : Exploiting Temporal Redundancy In Live Computer Vision Mark Buckler, Philip Bedoukian, Suren Jayasuriya, Adrian Sampson International Symposium on Computer Architecture (ISCA) Tuesday June 5, 2018
51 Backup Slides 51
52 Why not use vectors from video codec/isp? We ve demonstrated that the ISP can be skipped (Bucker et al. 2017) No need to compress video which is instantly thrown away Can save energy by power gating the ISP Opportunity to set own key frame schedule However, great idea for pre-stored video! 52
53 Why Not Simply Subsample? If lower frame rate needed, simply apply AMC at that frame rate Warping Adaptive key frame choice 53
54 Different Motion Estimation Methods Faster16 FasterM 54
55 Difference from Deep Feature Flow? Deep Feature Flow does also exploit temporal redundancy, but AMC and EVA 2 Deep Feature Flow Adaptive key frame rate? Yes No On chip activation cache? Yes No Learned motion estimation? No Yes Motion estimation granularity Per receptive field Per pixel (excess granularity) Motion compensation Sparse (four-way zero skip) Dense Activation storage Sparse (run length) Dense 55
56 Difference from Euphrates? Euphrates has a strong focus on SoC integration Motion estimation from ISP May want to skip the ISP to save energy & create more optimal key schedule Motion compensation on bounding boxes Skips entire network, but is only applicable to object detection 56
57 Re-use Tiles in RFBME 57
58 Changing Error Threshold 58
59 Different Adaptive Key Frame Metrics 59
60 Normalized Latency & Energy 60
61 How about Re-Training? 61
62 Where to cut the network? 62
63 #MakeRyanGoslingTheNewLenna Lenna dates back to 1973 We need a new test image for image processing!
EVA 2 : Exploiting Temporal Redundancy in Live Computer Vision
EVA 2 : Exploiting Temporal Redundancy in Live Computer Vision Mark Buckler Cornell University mab598@cornell.edu Philip Bedoukian Cornell University pbb59@cornell.edu Suren Jayasuriya Arizona State University
Research Faculty Summit Systems Fueling future disruptions
Research Faculty Summit 2018 Systems Fueling future disruptions Efficient Edge Computing for Deep Neural Networks and Beyond Vivienne Sze In collaboration with Yu-Hsin Chen, Joel Emer, Tien-Ju Yang, Sertac
Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks
Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks Yu-Hsin Chen 1, Joel Emer 1, 2, Vivienne Sze 1 1 MIT 2 NVIDIA 1 Contributions of This Work A novel energy-efficient
Towards a Uniform Template-based Architecture for Accelerating 2D and 3D CNNs on FPGA
Towards a Uniform Template-based Architecture for Accelerating 2D and 3D CNNs on FPGA Junzhong Shen, You Huang, Zelong Wang, Yuran Qiao, Mei Wen, Chunyuan Zhang National University of Defense Technology,
Deep Learning Accelerators
Deep Learning Accelerators Abhishek Srivastava (as29) Samarth Kulshreshtha (samarth5) University of Illinois, Urbana-Champaign Submitted as a requirement for CS 433 graduate student project Outline Introduction
Scalable and Modularized RTL Compilation of Convolutional Neural Networks onto FPGA
Scalable and Modularized RTL Compilation of Convolutional Neural Networks onto FPGA Yufei Ma, Naveen Suda, Yu Cao, Jae-sun Seo, Sarma Vrudhula School of Electrical, Computer and Energy Engineering School
Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAs
Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAs Ritchie Zhao 1, Weinan Song 2, Wentao Zhang 2, Tianwei Xing 3, Jeng-Hau Lin 4, Mani Srivastava 3, Rajesh Gupta 4, Zhiru
arxiv: v1 [cs.cv] 11 Feb 2018
![arxiv: v1 [cs.cv] 11 Feb 2018](/thumbs/79/79222664.jpg "arxiv: v1 [cs.cv] 11 Feb 2018")
arxiv:8.8v [cs.cv] Feb 8 - Partitioning of Deep Neural Networks with Feature Space Encoding for Resource-Constrained Internet-of-Things Platforms ABSTRACT Jong Hwan Ko, Taesik Na, Mohammad Faisal Amir,
Switched by Input: Power Efficient Structure for RRAMbased Convolutional Neural Network
Switched by Input: Power Efficient Structure for RRAMbased Convolutional Neural Network Lixue Xia, Tianqi Tang, Wenqin Huangfu, Ming Cheng, Xiling Yin, Boxun Li, Yu Wang, Huazhong Yang Dept. of E.E., Tsinghua
A Method to Estimate the Energy Consumption of Deep Neural Networks
A Method to Estimate the Consumption of Deep Neural Networks Tien-Ju Yang, Yu-Hsin Chen, Joel Emer, Vivienne Sze Massachusetts Institute of Technology, Cambridge, MA, USA {tjy, yhchen, jsemer, sze}@mit.edu
Bandwidth-Centric Deep Learning Processing through Software-Hardware Co-Design
Bandwidth-Centric Deep Learning Processing through Software-Hardware Co-Design Song Yao 姚颂 Founder & CEO DeePhi Tech 深鉴科技 song.yao@deephi.tech Outline - About DeePhi Tech - Background - Bandwidth Matters
Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks
Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks Naveen Suda, Vikas Chandra *, Ganesh Dasika *, Abinash Mohanty, Yufei Ma, Sarma Vrudhula, Jae-sun Seo, Yu
DNN Accelerator Architectures
DNN Accelerator Architectures ISCA Tutorial (2017) Website: http://eyeriss.mit.edu/tutorial.html Joel Emer, Vivienne Sze, Yu-Hsin Chen 1 2 Highly-Parallel Compute Paradigms Temporal Architecture (SIMD/SIMT)
Frequency Domain Acceleration of Convolutional Neural Networks on CPU-FPGA Shared Memory System
Frequency Domain Acceleration of Convolutional Neural Networks on CPU-FPGA Shared Memory System Chi Zhang, Viktor K Prasanna University of Southern California {zhan527, prasanna}@usc.edu fpga.usc.edu ACM
How to Estimate the Energy Consumption of Deep Neural Networks
How to Estimate the Energy Consumption of Deep Neural Networks Tien-Ju Yang, Yu-Hsin Chen, Joel Emer, Vivienne Sze MIT 1 Problem of DNNs Recognition Smart Drone AI Computation DNN 15k 300k OP/Px DPM 0.1k
CapsAcc: An Efficient Hardware Accelerator for CapsuleNets with Data Reuse
Accepted for publication at Design, Automation and Test in Europe (DATE 2019). Florence, Italy CapsAcc: An Efficient Hardware Accelerator for CapsuleNets with Reuse Alberto Marchisio, Muhammad Abdullah
TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory
TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory Mingyu Gao, Jing Pu, Xuan Yang, Mark Horowitz, Christos Kozyrakis Stanford University Platform Lab Review Feb 2017 Deep Neural
Computer Architectures for Deep Learning. Ethan Dell and Daniyal Iqbal
Computer Architectures for Deep Learning Ethan Dell and Daniyal Iqbal Agenda Introduction to Deep Learning Challenges Architectural Solutions Hardware Architectures CPUs GPUs Accelerators FPGAs SOCs ASICs
Maximizing Server Efficiency from μarch to ML accelerators. Michael Ferdman
Maximizing Server Efficiency from μarch to ML accelerators Michael Ferdman Maximizing Server Efficiency from μarch to ML accelerators Michael Ferdman Maximizing Server Efficiency with ML accelerators Michael
Flow-Based Video Recognition
Flow-Based Video Recognition Jifeng Dai Visual Computing Group, Microsoft Research Asia Joint work with Xizhou Zhu*, Yuwen Xiong*, Yujie Wang*, Lu Yuan and Yichen Wei (* interns) Talk pipeline Introduction
Instruction Driven Cross-Layer CNN Accelerator with Winograd Transformation on FPGA
Instruction Driven Cross-Layer CNN Accelerator with Winograd Transformation on FPGA Abstract In recent years, Convolutional Neural Network (CNN) has been widely applied in computer vision tasks. FPGAs
USING DATAFLOW TO OPTIMIZE ENERGY EFFICIENCY OF DEEP NEURAL NETWORK ACCELERATORS
... USING DATAFLOW TO OPTIMIZE ENERGY EFFICIENCY OF DEEP NEURAL NETWORK ACCELERATORS... Yu-Hsin Chen Massachusetts Institute of Technology Joel Emer Nvidia and Massachusetts Institute of Technology Vivienne
Design Space Exploration of FPGA-Based Deep Convolutional Neural Networks
Design Space Exploration of FPGA-Based Deep Convolutional Neural Networks Abstract Deep Convolutional Neural Networks (DCNN) have proven to be very effective in many pattern recognition applications, such
Towards Closing the Energy Gap Between HOG and CNN Features for Embedded Vision
Towards Closing the Energy Gap Between HOG and CNN Features for Embedded Vision The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation
Scaling Neural Network Acceleration using Coarse-Grained Parallelism
Scaling Neural Network Acceleration using Coarse-Grained Parallelism Mingyu Gao, Xuan Yang, Jing Pu, Mark Horowitz, Christos Kozyrakis Stanford University Platform Lab Review Feb 2018 Neural Networks (NNs)
Efficient Methods for Deep Learning
Efficient Methods for Deep Learning Song Han Stanford University Sep 2016 Background: Deep Learning for Everything Source: Brody Huval et al., An Empirical Evaluation, arxiv:1504.01716 Source: leon A.
direct hardware mapping of cnns on fpga-based smart cameras
direct hardware mapping of cnns on fpga-based smart cameras Workshop on Architecture of Smart Cameras Kamel ABDELOUAHAB, Francois BERRY, Maxime PELCAT, Jocelyn SEROT, Jean-Charles QUINTON Cordoba, June
Implementing Long-term Recurrent Convolutional Network Using HLS on POWER System
Implementing Long-term Recurrent Convolutional Network Using HLS on POWER System Xiaofan Zhang1, Mohamed El Hadedy1, Wen-mei Hwu1, Nam Sung Kim1, Jinjun Xiong2, Deming Chen1 1 University of Illinois Urbana-Champaign
Lecture 12: Model Serving. CSE599W: Spring 2018
Lecture 12: Model Serving CSE599W: Spring 2018 Deep Learning Applications That drink will get you to 2800 calories for today I last saw your keys in the store room Remind Tom of the party You re on page
ECE5775 High-Level Digital Design Automation, Fall 2018 School of Electrical Computer Engineering, Cornell University
ECE5775 High-Level Digital Design Automation, Fall 2018 School of Electrical Computer Engineering, Cornell University Lab 4: Binarized Convolutional Neural Networks Due Wednesday, October 31, 2018, 11:59pm
Revolutionizing the Datacenter
Power-Efficient Machine Learning using FPGAs on POWER Systems Ralph Wittig, Distinguished Engineer Office of the CTO, Xilinx Revolutionizing the Datacenter Join the Conversation #OpenPOWERSummit Top-5
Chain-NN: An Energy-Efficient 1D Chain Architecture for Accelerating Deep Convolutional Neural Networks
Chain-NN: An Energy-Efficient D Chain Architecture for Accelerating Deep Convolutional Neural Networks Shihao Wang, Dajiang Zhou, Xushen Han, Takeshi Yoshimura Graduate School of Information, Production
Design Space Exploration of FPGA-Based Deep Convolutional Neural Networks
Design Space Exploration of FPGA-Based Deep Convolutional Neural Networks Mohammad Motamedi, Philipp Gysel, Venkatesh Akella and Soheil Ghiasi Electrical and Computer Engineering Department, University
Deep learning for dense per-pixel prediction. Chunhua Shen The University of Adelaide, Australia
Deep learning for dense per-pixel prediction Chunhua Shen The University of Adelaide, Australia Image understanding Classification error Convolution Neural Networks 0.3 0.2 0.1 Image Classification [Krizhevsky
SmartShuttle: Optimizing Off-Chip Memory Accesses for Deep Learning Accelerators
SmartShuttle: Optimizing Off-Chip emory Accesses for Deep Learning Accelerators Jiajun Li, Guihai Yan, Wenyan Lu, Shuhao Jiang, Shijun Gong, Jingya Wu, Xiaowei Li State Key Laboratory of Computer Architecture,
arxiv: v2 [cs.cv] 3 May 2016
![arxiv: v2 [cs.cv] 3 May 2016](/thumbs/72/66717655.jpg "arxiv: v2 [cs.cv] 3 May 2016")
EIE: Efficient Inference Engine on Compressed Deep Neural Network Song Han Xingyu Liu Huizi Mao Jing Pu Ardavan Pedram Mark A. Horowitz William J. Dally Stanford University, NVIDIA {songhan,xyl,huizi,jingpu,perdavan,horowitz,dally}@stanford.edu
Software Defined Hardware
Software Defined Hardware For data intensive computation Wade Shen DARPA I2O September 19, 2017 1 Goal Statement Build runtime reconfigurable hardware and software that enables near ASIC performance (within
Lab 4: Convolutional Neural Networks Due Friday, November 3, 2017, 11:59pm
ECE5775 High-Level Digital Design Automation, Fall 2017 School of Electrical Computer Engineering, Cornell University Lab 4: Convolutional Neural Networks Due Friday, November 3, 2017, 11:59pm 1 Introduction
M.Tech Student, Department of ECE, S.V. College of Engineering, Tirupati, India
International Journal of Scientific Research in Computer Science, Engineering and Information Technology 2018 IJSRCSEIT Volume 3 Issue 5 ISSN : 2456-3307 High Performance Scalable Deep Learning Accelerator
Redundancy-reduced MobileNet Acceleration on Reconfigurable Logic For ImageNet Classification
Redundancy-reduced MobileNet Acceleration on Reconfigurable Logic For ImageNet Classification Jiang Su, Julian Faraone, Junyi Liu, Yiren Zhao, David B. Thomas, Philip H. W. Leong, and Peter Y. K. Cheung
Semantic Segmentation
Semantic Segmentation UCLA:https://goo.gl/images/I0VTi2 OUTLINE Semantic Segmentation Why? Paper to talk about: Fully Convolutional Networks for Semantic Segmentation. J. Long, E. Shelhamer, and T. Darrell,
Binary Convolutional Neural Network on RRAM
Binary Convolutional Neural Network on RRAM Tianqi Tang, Lixue Xia, Boxun Li, Yu Wang, Huazhong Yang Dept. of E.E, Tsinghua National Laboratory for Information Science and Technology (TNList) Tsinghua
Speculations about Computer Architecture in Next Three Years. Jan. 20, 2018
Speculations about Computer Architecture in Next Three Years shuchang.zhou@gmail.com Jan. 20, 2018 About me https://zsc.github.io/ Source-to-source transformation Cache simulation Compiler Optimization
High Performance Computing
High Performance Computing 9th Lecture 2016/10/28 YUKI ITO 1 Selected Paper: vdnn: Virtualized Deep Neural Networks for Scalable, MemoryEfficient Neural Network Design Minsoo Rhu, Natalia Gimelshein, Jason
BHNN: a Memory-Efficient Accelerator for Compressing Deep Neural Network with Blocked Hashing Techniques
BHNN: a Memory-Efficient Accelerator for Compressing Deep Neural Network with Blocked Hashing Techniques Jingyang Zhu 1, Zhiliang Qian 2*, and Chi-Ying Tsui 1 1 The Hong Kong University of Science and
DEEP LEARNING ACCELERATOR UNIT WITH HIGH EFFICIENCY ON FPGA
DEEP LEARNING ACCELERATOR UNIT WITH HIGH EFFICIENCY ON FPGA J.Jayalakshmi 1, S.Ali Asgar 2, V.Thrimurthulu 3 1 M.tech Student, Department of ECE, Chadalawada Ramanamma Engineering College, Tirupati Email
Lecture: Deep Convolutional Neural Networks
Lecture: Deep Convolutional Neural Networks Shubhang Desai Stanford Vision and Learning Lab 1 Today s agenda Deep convolutional networks History of CNNs CNN dev Architecture search 2 Previously argmax
The OpenVX Computer Vision and Neural Network Inference
The OpenVX Computer and Neural Network Inference Standard for Portable, Efficient Code Radhakrishna Giduthuri Editor, OpenVX Khronos Group radha.giduthuri@amd.com @RadhaGiduthuri Copyright 2018 Khronos
Encoder-Decoder Networks for Semantic Segmentation. Sachin Mehta
Encoder-Decoder Networks for Semantic Segmentation Sachin Mehta Outline > Overview of Semantic Segmentation > Encoder-Decoder Networks > Results What is Semantic Segmentation? Input: RGB Image Output:
JOINT DETECTION AND SEGMENTATION WITH DEEP HIERARCHICAL NETWORKS. Zhao Chen Machine Learning Intern, NVIDIA
JOINT DETECTION AND SEGMENTATION WITH DEEP HIERARCHICAL NETWORKS Zhao Chen Machine Learning Intern, NVIDIA ABOUT ME 5th year PhD student in physics @ Stanford by day, deep learning computer vision scientist
FAST: A Framework to Accelerate Super- Resolution Processing on Compressed Videos
FAST: A Framework to Accelerate Super- Resolution Processing on Compressed Videos Zhengdong Zhang, Vivienne Sze Massachusetts Institute of Technology http://www.mit.edu/~sze/fast.html 1 Super-Resolution
Bandwidth-Efficient Deep Learning
1 Bandwidth-Efficient Deep Learning from Compression to Acceleration Song Han Assistant Professor, EECS Massachusetts Institute of Technology 2 AI is Changing Our Lives Self-Driving Car Machine Translation
Scalpel: Customizing DNN Pruning to the Underlying Hardware Parallelism
Scalpel: Customizing DNN Pruning to the Underlying Hardware Parallelism Jiecao Yu 1, Andrew Lukefahr 1, David Palframan 2, Ganesh Dasika 2, Reetuparna Das 1, Scott Mahlke 1 1 University of Michigan 2 ARM
Optimizing CNN-based Object Detection Algorithms on Embedded FPGA Platforms
Optimizing CNN-based Object Detection Algorithms on Embedded FPGA Platforms Ruizhe Zhao 1, Xinyu Niu 1, Yajie Wu 2, Wayne Luk 1, and Qiang Liu 3 1 Imperial College London {ruizhe.zhao15,niu.xinyu10,w.luk}@imperial.ac.uk
ACCELERATING DEEP CONVOLUTIONAL NETWORKS USING LOW-PRECISION AND SPARSITY. Ganesh Venkatesh, Eriko Nurvitadhi, Debbie Marr
ACCELERATING DEEP CONVOLUTIONAL NETWORKS USING LOW-PRECISION AND SPARSITY Ganesh Venkatesh, Eriko Nurvitadhi, Debbie Marr Accelerator Architecture Lab, Intel Corporation ABSTRACT We explore techniques
Layer-wise Performance Bottleneck Analysis of Deep Neural Networks
Layer-wise Performance Bottleneck Analysis of Deep Neural Networks Hengyu Zhao, Colin Weinshenker*, Mohamed Ibrahim*, Adwait Jog*, Jishen Zhao University of California, Santa Cruz, *The College of William
A Communication-Centric Approach for Designing Flexible DNN Accelerators
THEME ARTICLE: Hardware Acceleration A Communication-Centric Approach for Designing Flexible DNN Accelerators Hyoukjun Kwon, High computational demands of deep neural networks Ananda Samajdar, and (DNNs)
Specializing Hardware for Image Processing
Lecture 6: Specializing Hardware for Image Processing Visual Computing Systems So far, the discussion in this class has focused on generating efficient code for multi-core processors such as CPUs and GPUs.
Hide-and-Seek: Forcing a network to be Meticulous for Weakly-supervised Object and Action Localization
Hide-and-Seek: Forcing a network to be Meticulous for Weakly-supervised Object and Action Localization Krishna Kumar Singh and Yong Jae Lee University of California, Davis ---- Paper Presentation Yixian
SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks
SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks Angshuman Parashar Minsoo Rhu Anurag Mukkara Antonio Puglielli Rangharajan Venkatesan Brucek Khailany Joel Emer Stephen W. Keckler
Improving Energy Efficiency of Block-Matching Motion Estimation Using Dynamic Partial Reconfiguration
, pp.517-521 http://dx.doi.org/10.14257/astl.2015.1 Improving Energy Efficiency of Block-Matching Motion Estimation Using Dynamic Partial Reconfiguration Jooheung Lee 1 and Jungwon Cho 2, * 1 Dept. of
Leveraging MLC STT-RAM for Energy-efficient CNN Training
Memory Demand (GB) Access Bandwidth(GB/s) Normalized Accesses Leveraging for Energy-efficient CNN Training Hengyu Zhao and Jishen Zhao University of California, San Diego {h6zhao, jzhao}@eng.ucsd.edu ABSTRACT
A COST-EFFICIENT RESIDUAL PREDICTION VLSI ARCHITECTURE FOR H.264/AVC SCALABLE EXTENSION
A COST-EFFICIENT RESIDUAL PREDICTION VLSI ARCHITECTURE FOR H.264/AVC SCALABLE EXTENSION Yi-Hau Chen, Tzu-Der Chuang, Chuan-Yung Tsai, Yu-Jen Chen, and Liang-Gee Chen DSP/IC Design Lab., Graduate Institute
Profiling the Performance of Binarized Neural Networks. Daniel Lerner, Jared Pierce, Blake Wetherton, Jialiang Zhang
Profiling the Performance of Binarized Neural Networks Daniel Lerner, Jared Pierce, Blake Wetherton, Jialiang Zhang 1 Outline Project Significance Prior Work Research Objectives Hypotheses Testing Framework
Energy-Efficient CNN Implementation on a Deeply Pipelined FPGA Cluster
Energy-Efficient CNN Implementation on a Deeply Pipelined Cluster Chen Zhang 1, Di Wu, Jiayu Sun 1, Guangyu Sun 1,3, Guojie Luo 1,3, and Jason Cong 1,,3 1 Center for Energy-Efficient Computing and Applications,
PRIME: A Novel Processing-in-memory Architecture for Neural Network Computation in ReRAM-based Main Memory
Scalable and Energy-Efficient Architecture Lab (SEAL) PRIME: A Novel Processing-in-memory Architecture for Neural Network Computation in -based Main Memory Ping Chi *, Shuangchen Li *, Tao Zhang, Cong
arxiv: v1 [cs.ne] 23 Mar 2018
![arxiv: v1 [cs.ne] 23 Mar 2018](/thumbs/81/84223479.jpg "arxiv: v1 [cs.ne] 23 Mar 2018")
SqueezeNext: Hardware-Aware Neural Network Design arxiv:1803.10615v1 [cs.ne] 23 Mar 2018 Amir Gholami, Kiseok Kwon, Bichen Wu, Zizheng Tai, Xiangyu Yue, Peter Jin, Sicheng Zhao, Kurt Keutzer EECS, UC Berkeley
Deploying Deep Neural Networks in the Embedded Space
Deploying Deep Neural Networks in the Embedded Space Stylianos I. Venieris, Alexandros Kouris, Christos-Savvas Bouganis 2 nd International Workshop on Embedded and Mobile Deep Learning (EMDL) MobiSys,
Predicting ground-level scene Layout from Aerial imagery. Muhammad Hasan Maqbool
Predicting ground-level scene Layout from Aerial imagery Muhammad Hasan Maqbool Objective Given the overhead image predict its ground level semantic segmentation Predicted ground level labeling Overhead/Aerial
Real-time object detection towards high power efficiency
Real-time object detection towards high power efficiency Jincheng Yu, Kaiyuan Guo, Yiming Hu, Xuefei Ning, Jiantao Qiu, Huizi Mao, Song Yao, Tianqi Tang, Boxun Li, Yu Wang, and Huazhong Yang Tsinghua University,
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun Presented by Tushar Bansal Objective 1. Get bounding box for all objects
Hermes: An Integrated CPU/GPU Microarchitecture for IP Routing
Hermes: An Integrated CPU/GPU Microarchitecture for IP Routing Yuhao Zhu * Yangdong Deng Yubei Chen * Electrical and Computer Engineering University of Texas at Austin Institute of Microelectronics Tsinghua
Neural Network-Hardware Co-design for Scalable RRAM-based BNN Accelerators
Neural Network-Hardware Co-design for Scalable RRAM-based BNN Accelerators Yulhwa Kim, Hyungjun Kim, and Jae-Joon Kim Dept. of Creative IT Engineering, Pohang University of Science and Technology (POSTECH),
Real-time convolutional networks for sonar image classification in low-power embedded systems
Real-time convolutional networks for sonar image classification in low-power embedded systems Matias Valdenegro-Toro Ocean Systems Laboratory - School of Engineering & Physical Sciences Heriot-Watt University,
Exploring Heterogeneous Algorithms for Accelerating Deep Convolutional Neural Networks on FPGAs
Exploring Heterogeneous Algorithms for Accelerating Deep Convolutional Neural Networks on FPGAs Qingcheng Xiao *1, Yun Liang 1, Liqiang Lu 1, Shengen Yan,3 and Yu-Wing Tai 3 1 Center for Energy-efficient
Two FPGA-DNN Projects: 1. Low Latency Multi-Layer Perceptrons using FPGAs 2. Acceleration of CNN Training on FPGA-based Clusters
Two FPGA-DNN Projects: 1. Low Latency Multi-Layer Perceptrons using FPGAs 2. Acceleration of CNN Training on FPGA-based Clusters *Argonne National Lab +BU & USTC Presented by Martin Herbordt Work by Ahmed
Comparing Memory Systems for Chip Multiprocessors
Comparing Memory Systems for Chip Multiprocessors Jacob Leverich Hideho Arakida, Alex Solomatnikov, Amin Firoozshahian, Mark Horowitz, Christos Kozyrakis Computer Systems Laboratory Stanford University
Photo-realistic Renderings for Machines Seong-heum Kim
Photo-realistic Renderings for Machines 20105034 Seong-heum Kim CS580 Student Presentations 2016.04.28 Photo-realistic Renderings for Machines Scene radiances Model descriptions (Light, Shape, Material,
ESE: Efficient Speech Recognition Engine for Sparse LSTM on FPGA
ESE: Efficient Speech Recognition Engine for Sparse LSTM on FPGA Song Han 1,2, Junlong Kang 2, Huizi Mao 1, Yiming Hu 3, Xin Li 2, Yubin Li 2, Dongliang Xie 2, Hong Luo 2, Song Yao 2, Yu Wang 2,3, Huazhong
Bit-Pragmatic Deep Neural Network Computing
Bit-Pragmatic Deep Neural Network Computing Jorge Albericio, Patrick Judd, Alberto Delmás, Sayeh Sharify, Andreas Moshovos Department of Electrical and Computer Engineering University of Toronto {jorge,
Deep learning for object detection. Slides from Svetlana Lazebnik and many others
Deep learning for object detection Slides from Svetlana Lazebnik and many others Recent developments in object detection 80% PASCAL VOC mean0average0precision0(map) 70% 60% 50% 40% 30% 20% 10% Before deep
RTSR: Enhancing Real-time H.264 Video Streaming using Deep Learning based Video Super Resolution Spring 2017 CS570 Project Presentation June 8, 2017
RTSR: Enhancing Real-time H.264 Video Streaming using Deep Learning based Video Super Resolution Spring 2017 CS570 Project Presentation June 8, 2017 Team 16 Soomin Kim Leslie Tiong Youngki Kwon Insu Jang
Minimizing Computation in Convolutional Neural Networks
Minimizing Computation in Convolutional Neural Networks Jason Cong and Bingjun Xiao Computer Science Department, University of California, Los Angeles, CA 90095, USA {cong,xiao}@cs.ucla.edu Abstract. Convolutional
Co-synthesis and Accelerator based Embedded System Design
Co-synthesis and Accelerator based Embedded System Design COE838: Embedded Computer System http://www.ee.ryerson.ca/~courses/coe838/ Dr. Gul N. Khan http://www.ee.ryerson.ca/~gnkhan Electrical and Computer
THE NVIDIA DEEP LEARNING ACCELERATOR
THE NVIDIA DEEP LEARNING ACCELERATOR INTRODUCTION NVDLA NVIDIA Deep Learning Accelerator Developed as part of Xavier NVIDIA s SOC for autonomous driving applications Optimized for Convolutional Neural
CMU Lecture 18: Deep learning and Vision: Convolutional neural networks. Teacher: Gianni A. Di Caro
CMU 15-781 Lecture 18: Deep learning and Vision: Convolutional neural networks Teacher: Gianni A. Di Caro DEEP, SHALLOW, CONNECTED, SPARSE? Fully connected multi-layer feed-forward perceptrons: More powerful
DNN ENGINE: A 16nm Sub-uJ DNN Inference Accelerator for the Embedded Masses
DNN ENGINE: A 16nm Sub-uJ DNN Inference Accelerator for the Embedded Masses Paul N. Whatmough 1,2 S. K. Lee 2, N. Mulholland 2, P. Hansen 2, S. Kodali 3, D. Brooks 2, G.-Y. Wei 2 1 ARM Research, Boston,
CS231N Section. Video Understanding 6/1/2018
CS231N Section Video Understanding 6/1/2018 Outline Background / Motivation / History Video Datasets Models Pre-deep learning CNN + RNN 3D convolution Two-stream What we ve seen in class so far... Image
Deep Incremental Scene Understanding. Federico Tombari & Christian Rupprecht Technical University of Munich, Germany
Deep Incremental Scene Understanding Federico Tombari & Christian Rupprecht Technical University of Munich, Germany C. Couprie et al. "Toward Real-time Indoor Semantic Segmentation Using Depth Information"
Optimize Deep Convolutional Neural Network with Ternarized Weights and High Accuracy
Optimize Deep Convolutional Neural Network with Ternarized Weights and High Zhezhi He Department of ECE University of Central Florida Elliot.he@knights.ucf.edu Boqing Gong Tencent AI Lab Bellevue, WA 98004
Deep Learning Requirements for Autonomous Vehicles
Deep Learning Requirements for Autonomous Vehicles Pierre Paulin, Director of R&D Synopsys Inc. Chipex, 1 May 2018 1 Agenda Deep Learning and Convolutional Neural Networks for Embedded Vision Automotive
Escher: A CNN Accelerator with Flexible Buffering to Minimize Off-Chip Transfer
Escher: A CNN Accelerator with Flexible Buffering to Minimize ff-chip Transfer Yongming Shen Stony Brook University yoshen@cs.stonybrook.edu Michael Ferdman Stony Brook University mferdman@cs.stonybrook.edu
Efficient Segmentation-Aided Text Detection For Intelligent Robots
Efficient Segmentation-Aided Text Detection For Intelligent Robots Junting Zhang, Yuewei Na, Siyang Li, C.-C. Jay Kuo University of Southern California Outline Problem Definition and Motivation Related
DPS: Dynamic Precision Scaling for Stochastic Computing-based Deep Neural Networks
DPS: Dynamic Precision Scaling for Stochastic Computing-based Deep Neural Networks Hyeonuk Sim, Saken Kenzhegulov, and Jongeun Lee School of Electrical and Computer Engineering, UNIST, Ulsan, 99 South
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. By Joa õ Carreira and Andrew Zisserman Presenter: Zhisheng Huang 03/02/2018
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset By Joa õ Carreira and Andrew Zisserman Presenter: Zhisheng Huang 03/02/2018 Outline: Introduction Action classification architectures
DRAF: A Low-Power DRAM-based Reconfigurable Acceleration Fabric
DRAF: A Low-Power DRAM-based Reconfigurable Acceleration Fabric Mingyu Gao, Christina Delimitrou, Dimin Niu, Krishna Malladi, Hongzhong Zheng, Bob Brennan, Christos Kozyrakis ISCA June 22, 2016 FPGA-Based
PARTIAL STYLE TRANSFER USING WEAKLY SUPERVISED SEMANTIC SEGMENTATION. Shin Matsuo Wataru Shimoda Keiji Yanai
PARTIAL STYLE TRANSFER USING WEAKLY SUPERVISED SEMANTIC SEGMENTATION Shin Matsuo Wataru Shimoda Keiji Yanai Department of Informatics, The University of Electro-Communications, Tokyo 1-5-1 Chofugaoka,
Scene Text Recognition for Augmented Reality. Sagar G V Adviser: Prof. Bharadwaj Amrutur Indian Institute Of Science
Scene Text Recognition for Augmented Reality Sagar G V Adviser: Prof. Bharadwaj Amrutur Indian Institute Of Science Outline Research area and motivation Finding text in natural scenes Prior art Improving
Index. Springer Nature Switzerland AG 2019 B. Moons et al., Embedded Deep Learning,
Index A Algorithmic noise tolerance (ANT), 93 94 Application specific instruction set processors (ASIPs), 115 116 Approximate computing application level, 95 circuits-levels, 93 94 DAS and DVAS, 107 110
GeForce4. John Montrym Henry Moreton
GeForce4 John Montrym Henry Moreton 1 Architectural Drivers Programmability Parallelism Memory bandwidth 2 Recent History: GeForce 1&2 First integrated geometry engine & 4 pixels/clk Fixed-function transform,