Hardware Acceleration of Feature Detection and Description Algorithms on Low Power Embedded Platforms

Size: px
Start display at page:

Download "Hardware Acceleration of Feature Detection and Description Algorithms on Low Power Embedded Platforms"

Transcription

1 Hardware Acceleration of Feature Detection and Description Algorithms on LowPower Embedded Platforms Onur Ulusel, Christopher Picardo, Christopher Harris, Sherief Reda, R. Iris Bahar, School of Engineering, Brown University

2 Image Processing in Mobile Systems Image processing is everywhere! Input data has changed from words/numbers to images Sensors have improved dramatically Image processing is a major driving factor in technological advancement Autonomization relies on image processing Mobile/Embedded platforms?? Realtime computing limited data bandwidth prefer local computing to offloading to cloud BUT image processing can be very computationally intensive and power hungry 2

3 Accelerating Image Processing on Low Power Embedded Platforms Meeting real time image processing requirements for many of these applications requires HW assisted acceleration Which algorithms do we accelerate? Feature detection and feature description are key building blocks for image retrieval, biometric identification, visual odometry, etc. Computational efficient detection and analysis of image features is critical for performance and energyefficiency

4 Hardware Acceleration for Energy Constrained Image Processing Low power embedded platforms Field Programmable Gate Arrays (FPGAs) Graphical Processing Units (GPUs) Low power general processors (CPUs) Xilinx Virtex 6 FPGAs Xilinx Zynq core NVIDIA GeForce GTX 680 GPUs 192 core NVIDIA Jetson TK1 Power 15 W <5W 195 W <12W 4

5 Our Contributions Comparative study of feature detection and description algorithms What are their computation kernel characteristics? Comparative study of platforms for embedded applications Advantages/disadvantages of each platform? Accelerating algorithms on different platforms How can algorithms be modified to better exploit available hardware of each platform? How does performance compare in terms of run time and energy consumption? 5

6 What is a feature? Feature Detection An interesting part of an image that can be used to identify objects Examples: Edges, corners, ridges, blobs flat region: no change within block edge: no change along the edge corner: change in all directions Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institude 6

7 Feature Description Given the features, uniquely describe them so they can be matched in other images Descriptors summarize characteristics of the features E.g., intensity, orientation Descriptors should be distinctive and insensitive to local image deformations. Images from: R. Szeliski, Computer Vision: Algorithms and Applications 7

8 Accuracy and Runtime Comparisons Accuracy Percetage SIFT SURF BRIEF BRISK HoG (Histogram of Gradient) based Descriptors SIFT: ScaleInvariant Feature Transform SURF: Speeded Up Robust Features Binary Feature Descriptors BRIEF: Binary Robust Independent Elementary Features BRISK: Binary Robust Invariant Scalable Keypoints 10 CPU runtime (ms) log scale Precision (% of detected features that are correct) Recall (% of features detected) Runtime 8

9 FAST: Features from Accelerated Segment Test 12pixel continuity? If so then feature Rosten and Drummond, ECCV 06 Precompare pixels 1, 5, 9, and 13 to determine possibility for continuity On average 98.5% of the comparisons fail the continuity test at the precompare stage sliding window Bresenham Circle 9

10 BRIEF: Binary Robust Independent Elementary Features Compare intensities of pairs of points using Hamming distance BRIEF Sampling pattern 512 sampling pairs For each pair, X i is at (0,0) and Y i takes all possible values from coarse polar grid Chosen sampling pattern Sampling pairs are generated from a results in a 512bit region around center pixel characterization array 10

11 BRISK: Binary Robust Invariant Scalable Keypoints BRISK uses custom sampling pattern 512 sampling pairs generated from a region (like BRIEF) Distinguishes between short/long pairs Short pairs used similar to BRIEF to generate descriptor vectors based on intensity comparisons Long pairs used for orientation computation by rotating sampling pattern BRISK sampling pattern Red circles represent standard deviation of Gaussian smoothing 11

12 Start Read Input Frame Algorithm Flowchart FAST BRIEF For each pixel, p, apply the 7x7 filter of Bresenham circle p is a corner Generate N sampling pairs, X i and Y i, around p X i > Y i Feature Detection False D i = 0 False FAST feature detection BRIEF feature description Obtaining sampling window for feature description requires irregular access pattern D i = 1 Brief Feature Description Stop

13 Start Read Input Frame Algorithm Flowchart FAST BRISK For each pixel, p, apply the 7x7 filter of Bresenham circle p is a corner Generate N sampling pairs, X i and Y i, around p X i > Y i Feature Detection False False D i = 0 FAST feature detection BRISK feature description BRISK requires an extra step for orientation compensation A significant amount of extra hardware resources for this step D i = 1 Perform Orientation Compensation for BRISK Stop Feature Description

14 Experimental Embedded Platforms FPGA: MicroZED development board: 28nm Zynq 7020 SoC Artix7 FPGA 1GB DDR3 dualcore Arm Cortex A9 CPU (for debug and init. only) GPU & CPU: Jetson TK1 development kit 28nm Tegra K1 SoC Kepler GPU with 192 CUDA 950MHz Quadcore ARM Cortex A15 2.5GHz (single core activated) 2GB Memory Running OpenCV versions of FAST, BRIEF, BRISK 14

15 Feature Detection & Description: Block Diagram DDR3 ARM Cortex A9 CPU Central Interconnect Memory Interface Processing System (PS) 32b GP AXI Master Port AXI Interconnect Control 10Line word Buffer Enable & Sync signals Image data Buffer Address Generator and Register Array ZigZag Traversing Line Buffers X x x x Precompute units Equality Mask Size Register Array X 12 Circle Comparator Programmable Logic (PL) is_corner Smoothing & Region Generation Nwide comparator Orientation Compensation x&y coordinates descriptor orientation 15

16 Feature Detection & Description: Block Diagram DDR3 ARM Cortex A9 CPU Central Interconnect Memory Interface Processing System (PS) 32b GP AXI Master Port AXI Interconnect Control 10Line word Buffer Enable & Sync signals Image data Buffer Address Generator and Register Array ZigZag Traversing Line Buffers X x x x Precompute units Equality Mask Size Register Array X 12 Circle Comparator Programmable Logic (PL) is_corner Smoothing & Region Generation Nwide comparator Orientation Compensation x&y coordinates descriptor orientation FAST Feature Detection 16

17 Feature Detection & Description: Block Diagram BRIEF Descriptor DDR3 ARM Cortex A9 CPU Central Interconnect Memory Interface Processing System (PS) 32b GP AXI Master Port AXI Interconnect Control 10Line word Buffer Enable & Sync signals Image data Buffer Address Generator and Register Array ZigZag Traversing Line Buffers X x x x Precompute units Equality Mask Size Register Array X 12 Circle Comparator Programmable Logic (PL) is_corner Smoothing & Region Generation Nwide comparator Orientation Compensation x&y coordinates descriptor orientation 17

18 Feature Detection & Description: Block Diagram BRISK Descriptor DDR3 ARM Cortex A9 CPU Central Interconnect Memory Interface Processing System (PS) 32b GP AXI Master Port AXI Interconnect Control 10Line word Buffer Enable & Sync signals Image data Buffer Address Generator and Register Array ZigZag Traversing Line Buffers X x x x Precompute units Equality Mask Size Register Array X 12 Circle Comparator Programmable Logic (PL) is_corner Smoothing & Region Generation Nwide comparator Orientation Compensation x&y coordinates descriptor orientation 18

19 Feature Detection & Description: Block Diagram DDR3 ARM Cortex A9 CPU Central Interconnect Memory Interface Processing System (PS) 32b GP AXI Master Port AXI Interconnect Control 10Line word Buffer Enable & Sync signals Image data Data control logic for detection and description Buffer Address Generator and Register Array ZigZag Traversing Line Buffers X x x x Precompute units Equality Mask Size Register Array X 12 Circle Comparator Programmable Logic (PL) is_corner Descriptor logic Smoothing & Region Generation Nwide comparator Orientation Compensation x&y coordinates descriptor orientation Feature detection logic 19

20 Results: Runtime Runtime (ms) Intel i7 CPU ARM on Jetson Tegra GPU on Jetson Zynq FPGA FAST FASTBRIEF FASTBRISK 20

21 Results: Power & Energy 25 (detection) (detection description) Intel i7 CPU Embedded CPU Tegra GPU Zynq FPGA 20 Power (W) FAST FASTBRIEF FASTBRISK Energy (mj) Intel i7 CPU Embedded CPU Tegra GPU Zynq FPGA FAST FASTBRIEF FASTBRISK

22 Results: Profiling Not Selected Stall Reasons during GPU Computation 100% 80% Instruction Distribution Memory Throttle 60% Pipe Busy 40% 20% Execution Dependency 0% FPGA: FAST FPGA: FAST BRIEF GPU: FAST GPU: FAST BRIEF 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% Description Detection (BRIEF) (FAST) Feature description stalled due to memory throttle Needs better data management Load/Store FP/Integer Feature description for GPU implementation has bump in load/store ops Almost 10X more than just FAST 22

23 Results: FPGA Resource Utilization Resource utilization Lookup Tables Flip Flops Block RAMs FAST FAST BRIEF FAST BRISK % 32% 57% Distribution of resources 19% 14% 66% 37% 37% 27% LookUp Tables Flip Flops Block Rams FAST FASTBRIEF FASTBRISK BRISK requires significant amount of extra resources for smoothing and orienting Extra resources do not translate to much extra power 23

24 Conclusions FPGA outperforms CPUs and GPUs in terms of power & performance FAST BRISK: 36 fps vs. 147 fps FPGA amenable to various HW optimizations: deep pipelining, optimized memory access, precomputation FGPA implementations better for handling multiple kernels For GPUs, multiple kernels highly bounded by kernel scheduler and memory bottlenecks FPGA customization on layers better for tackling operations on multiple kernels. Use profiling on GPU implementation as first step to FPGA optimization identify nature of bottlenecks Customized FPGA HW can often better manage certain types of bottlenecks 24

Hardware Acceleration of Feature Detection and Description Algorithms on Low-Power Embedded Platforms

Hardware Acceleration of Feature Detection and Description Algorithms on Low-Power Embedded Platforms Hardware Acceleration of Feature Detection and Description Algorithms on Low-Power Embedded Platforms Onur Ulusel, Christopher Picardo, Christopher B. Harris, Sherief Reda and R. Iris Bahar School of Engineering

More information

Deep Neural Network Enhanced VSLAM Landmark Selection

Deep Neural Network Enhanced VSLAM Landmark Selection Deep Neural Network Enhanced VSLAM Landmark Selection Dr. Patrick Benavidez Overview 1 Introduction 2 Background on methods used in VSLAM 3 Proposed Method 4 Testbed 5 Preliminary Results What is VSLAM?

More information

A hardware design of optimized ORB algorithm with reduced hardware cost

A hardware design of optimized ORB algorithm with reduced hardware cost , pp.58-62 http://dx.doi.org/10.14257/astl.2013 A hardware design of optimized ORB algorithm with reduced hardware cost Kwang-yeob Lee 1, Kyung-jin Byun 2 1 Dept. of Computer Engineering, Seokyenog University,

More information

Local features and image matching. Prof. Xin Yang HUST

Local features and image matching. Prof. Xin Yang HUST Local features and image matching Prof. Xin Yang HUST Last time RANSAC for robust geometric transformation estimation Translation, Affine, Homography Image warping Given a 2D transformation T and a source

More information

Efficient Interest Point Detectors & Features

Efficient Interest Point Detectors & Features Efficient Interest Point Detectors & Features Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Review. Efficient Interest Point Detectors. Efficient Descriptors. Review In classical

More information

OpenCV on Zynq: Accelerating 4k60 Dense Optical Flow and Stereo Vision. Kamran Khan, Product Manager, Software Acceleration and Libraries July 2017

OpenCV on Zynq: Accelerating 4k60 Dense Optical Flow and Stereo Vision. Kamran Khan, Product Manager, Software Acceleration and Libraries July 2017 OpenCV on Zynq: Accelerating 4k60 Dense Optical Flow and Stereo Vision Kamran Khan, Product Manager, Software Acceleration and Libraries July 2017 Agenda Why Zynq SoCs for Traditional Computer Vision Automated

More information

Computer Vision for HCI. Topics of This Lecture

Computer Vision for HCI. Topics of This Lecture Computer Vision for HCI Interest Points Topics of This Lecture Local Invariant Features Motivation Requirements, Invariances Keypoint Localization Features from Accelerated Segment Test (FAST) Harris Shi-Tomasi

More information

Image Features: Detection, Description, and Matching and their Applications

Image Features: Detection, Description, and Matching and their Applications Image Features: Detection, Description, and Matching and their Applications Image Representation: Global Versus Local Features Features/ keypoints/ interset points are interesting locations in the image.

More information

TR An Overview of NVIDIA Tegra K1 Architecture. Ang Li, Radu Serban, Dan Negrut

TR An Overview of NVIDIA Tegra K1 Architecture. Ang Li, Radu Serban, Dan Negrut TR-2014-17 An Overview of NVIDIA Tegra K1 Architecture Ang Li, Radu Serban, Dan Negrut November 20, 2014 Abstract This paperwork gives an overview of NVIDIA s Jetson TK1 Development Kit and its Tegra K1

More information

A Configurable Parallel Hardware Architecture for Efficient Integral Histogram Image Computing

A Configurable Parallel Hardware Architecture for Efficient Integral Histogram Image Computing A Configurable Parallel Hardware Architecture for Efficient Integral Histogram Image Computing Abstract: Integral histogram image can accelerate the computing process of feature algorithm in computer vision,

More information

Local invariant features

Local invariant features Local invariant features Tuesday, Oct 28 Kristen Grauman UT-Austin Today Some more Pset 2 results Pset 2 returned, pick up solutions Pset 3 is posted, due 11/11 Local invariant features Detection of interest

More information

Local Difference Binary for Ultrafast and Distinctive Feature Description

Local Difference Binary for Ultrafast and Distinctive Feature Description Local Difference Binary for Ultrafast and Distinctive Feature Description Xin Yang, K.-T. Tim Cheng IEEE Trans. on Pattern Analysis and Machine Intelligence, 2014, January *Source code has been released

More information

Computer Vision. Recap: Smoothing with a Gaussian. Recap: Effect of σ on derivatives. Computer Science Tripos Part II. Dr Christopher Town

Computer Vision. Recap: Smoothing with a Gaussian. Recap: Effect of σ on derivatives. Computer Science Tripos Part II. Dr Christopher Town Recap: Smoothing with a Gaussian Computer Vision Computer Science Tripos Part II Dr Christopher Town Recall: parameter σ is the scale / width / spread of the Gaussian kernel, and controls the amount of

More information

Lecture 4.1 Feature descriptors. Trym Vegard Haavardsholm

Lecture 4.1 Feature descriptors. Trym Vegard Haavardsholm Lecture 4.1 Feature descriptors Trym Vegard Haavardsholm Feature descriptors Histogram of Gradients (HoG) descriptors Binary descriptors 2 Histogram of Gradients (HOG) descriptors Scale Invariant Feature

More information

FPGA-Based Feature Detection

FPGA-Based Feature Detection FPGA-Based Feature Detection Wennie Tabib School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 wtabib@andrew.cmu.edu Abstract Fast, accurate, autonomous robot navigation is essential

More information

Harder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford

Harder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford Image matching Harder case by Diva Sian by Diva Sian by scgbt by swashford Even harder case Harder still? How the Afghan Girl was Identified by Her Iris Patterns Read the story NASA Mars Rover images Answer

More information

Image matching. Announcements. Harder case. Even harder case. Project 1 Out today Help session at the end of class. by Diva Sian.

Image matching. Announcements. Harder case. Even harder case. Project 1 Out today Help session at the end of class. by Diva Sian. Announcements Project 1 Out today Help session at the end of class Image matching by Diva Sian by swashford Harder case Even harder case How the Afghan Girl was Identified by Her Iris Patterns Read the

More information

Efficient Interest Point Detectors & Features

Efficient Interest Point Detectors & Features Efficient Interest Point Detectors & Features Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Review. Efficient Interest Point Detectors. Efficient Descriptors. Review In classical

More information

The rcuda middleware and applications

The rcuda middleware and applications The rcuda middleware and applications Will my application work with rcuda? rcuda currently provides binary compatibility with CUDA 5.0, virtualizing the entire Runtime API except for the graphics functions,

More information

INTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES. Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp.

INTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES. Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp. INTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp. Computer Vision in Mobile Tegra K1 It s time! AGENDA Use cases categories

More information

CS 378: Autonomous Intelligent Robotics. Instructor: Jivko Sinapov

CS 378: Autonomous Intelligent Robotics. Instructor: Jivko Sinapov CS 378: Autonomous Intelligent Robotics Instructor: Jivko Sinapov http://www.cs.utexas.edu/~jsinapov/teaching/cs378/ Visual Registration and Recognition Announcements Homework 6 is out, due 4/5 4/7 Installing

More information

EECS150 - Digital Design Lecture 14 FIFO 2 and SIFT. Recap and Outline

EECS150 - Digital Design Lecture 14 FIFO 2 and SIFT. Recap and Outline EECS150 - Digital Design Lecture 14 FIFO 2 and SIFT Oct. 15, 2013 Prof. Ronald Fearing Electrical Engineering and Computer Sciences University of California, Berkeley (slides courtesy of Prof. John Wawrzynek)

More information

Manycore and GPU Channelisers. Seth Hall High Performance Computing Lab, AUT

Manycore and GPU Channelisers. Seth Hall High Performance Computing Lab, AUT Manycore and GPU Channelisers Seth Hall High Performance Computing Lab, AUT GPU Accelerated Computing GPU-accelerated computing is the use of a graphics processing unit (GPU) together with a CPU to accelerate

More information

Embedded real-time stereo estimation via Semi-Global Matching on the GPU

Embedded real-time stereo estimation via Semi-Global Matching on the GPU Embedded real-time stereo estimation via Semi-Global Matching on the GPU Daniel Hernández Juárez, Alejandro Chacón, Antonio Espinosa, David Vázquez, Juan Carlos Moure and Antonio M. López Computer Architecture

More information

Harder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford

Harder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford Image matching Harder case by Diva Sian by Diva Sian by scgbt by swashford Even harder case Harder still? How the Afghan Girl was Identified by Her Iris Patterns Read the story NASA Mars Rover images Answer

More information

Local features: detection and description. Local invariant features

Local features: detection and description. Local invariant features Local features: detection and description Local invariant features Detection of interest points Harris corner detection Scale invariant blob detection: LoG Description of local patches SIFT : Histograms

More information

GPU Accelerating Speeded-Up Robust Features Timothy B. Terriberry, Lindley M. French, and John Helmsen

GPU Accelerating Speeded-Up Robust Features Timothy B. Terriberry, Lindley M. French, and John Helmsen GPU Accelerating Speeded-Up Robust Features Timothy B. Terriberry, Lindley M. French, and John Helmsen Overview of ArgonST Manufacturer of integrated sensor hardware and sensor analysis systems 2 RF, COMINT,

More information

Local Image preprocessing (cont d)

Local Image preprocessing (cont d) Local Image preprocessing (cont d) 1 Outline - Edge detectors - Corner detectors - Reading: textbook 5.3.1-5.3.5 and 5.3.10 2 What are edges? Edges correspond to relevant features in the image. An edge

More information

Click to edit title style

Click to edit title style Class 2: Low-level Representation Liangliang Cao, Jan 31, 2013 EECS 6890 Topics in Information Processing Spring 2013, Columbia University http://rogerioferis.com/visualrecognitionandsearch Visual Recognition

More information

High Quality Real Time Image Processing Framework on Mobile Platforms using Tegra K1. Eyal Hirsch

High Quality Real Time Image Processing Framework on Mobile Platforms using Tegra K1. Eyal Hirsch High Quality Real Time Image Processing Framework on Mobile Platforms using Tegra K1 Eyal Hirsch Established in 2009 and headquartered in Israel SagivTech Snapshot Core domain expertise: GPU Computing

More information

Edge and corner detection

Edge and corner detection Edge and corner detection Prof. Stricker Doz. G. Bleser Computer Vision: Object and People Tracking Goals Where is the information in an image? How is an object characterized? How can I find measurements

More information

Zynq Ultrascale+ Architecture

Zynq Ultrascale+ Architecture Zynq Ultrascale+ Architecture Stephanie Soldavini and Andrew Ramsey CMPE-550 Dec 2017 Soldavini, Ramsey (CMPE-550) Zynq Ultrascale+ Architecture Dec 2017 1 / 17 Agenda Heterogeneous Computing Zynq Ultrascale+

More information

CS 4495 Computer Vision A. Bobick. CS 4495 Computer Vision. Features 2 SIFT descriptor. Aaron Bobick School of Interactive Computing

CS 4495 Computer Vision A. Bobick. CS 4495 Computer Vision. Features 2 SIFT descriptor. Aaron Bobick School of Interactive Computing CS 4495 Computer Vision Features 2 SIFT descriptor Aaron Bobick School of Interactive Computing Administrivia PS 3: Out due Oct 6 th. Features recap: Goal is to find corresponding locations in two images.

More information

International Journal Of Global Innovations -Vol.6, Issue.I Paper Id: SP-V6-I1-P01 ISSN Online:

International Journal Of Global Innovations -Vol.6, Issue.I Paper Id: SP-V6-I1-P01 ISSN Online: IMPLEMENTATION OF OBJECT RECOGNITION USING SIFT ALGORITHM ON BEAGLE BOARD XM USING EMBEDDED LINUX #1 T.KRISHNA KUMAR -M. Tech Student, #2 G.SUDHAKAR - Assistant Professor, #3 R. MURALI, HOD - Assistant

More information

Feature descriptors and matching

Feature descriptors and matching Feature descriptors and matching Detections at multiple scales Invariance of MOPS Intensity Scale Rotation Color and Lighting Out-of-plane rotation Out-of-plane rotation Better representation than color:

More information

Zynq-7000 All Programmable SoC Product Overview

Zynq-7000 All Programmable SoC Product Overview Zynq-7000 All Programmable SoC Product Overview The SW, HW and IO Programmable Platform August 2012 Copyright 2012 2009 Xilinx Introducing the Zynq -7000 All Programmable SoC Breakthrough Processing Platform

More information

Simplify System Complexity

Simplify System Complexity Simplify System Complexity With the new high-performance CompactRIO controller Fanie Coetzer Field Sales Engineer Northern South Africa 2 3 New control system CompactPCI MMI/Sequencing/Logging FieldPoint

More information

AK Computer Vision Feature Point Detectors and Descriptors

AK Computer Vision Feature Point Detectors and Descriptors AK Computer Vision Feature Point Detectors and Descriptors 1 Feature Point Detectors and Descriptors: Motivation 2 Step 1: Detect local features should be invariant to scale and rotation, or perspective

More information

Image Features. Work on project 1. All is Vanity, by C. Allan Gilbert,

Image Features. Work on project 1. All is Vanity, by C. Allan Gilbert, Image Features Work on project 1 All is Vanity, by C. Allan Gilbert, 1873-1929 Feature extrac*on: Corners and blobs c Mo*va*on: Automa*c panoramas Credit: Ma9 Brown Why extract features? Mo*va*on: panorama

More information

CS4670: Computer Vision

CS4670: Computer Vision CS4670: Computer Vision Noah Snavely Lecture 6: Feature matching and alignment Szeliski: Chapter 6.1 Reading Last time: Corners and blobs Scale-space blob detector: Example Feature descriptors We know

More information

The SIFT (Scale Invariant Feature

The SIFT (Scale Invariant Feature The SIFT (Scale Invariant Feature Transform) Detector and Descriptor developed by David Lowe University of British Columbia Initial paper ICCV 1999 Newer journal paper IJCV 2004 Review: Matt Brown s Canonical

More information

Implementation of GP-GPU with SIMT Architecture in the Embedded Environment

Implementation of GP-GPU with SIMT Architecture in the Embedded Environment , pp.221-226 http://dx.doi.org/10.14257/ijmue.2014.9.4.23 Implementation of GP-GPU with SIMT Architecture in the Embedded Environment Kwang-yeob Lee and Jae-chang Kwak 1 * Dept. of Computer Engineering,

More information

Local Features: Detection, Description & Matching

Local Features: Detection, Description & Matching Local Features: Detection, Description & Matching Lecture 08 Computer Vision Material Citations Dr George Stockman Professor Emeritus, Michigan State University Dr David Lowe Professor, University of British

More information

A Lightweight YOLOv2:

A Lightweight YOLOv2: FPGA2018 @Monterey A Lightweight YOLOv2: A Binarized CNN with a Parallel Support Vector Regression for an FPGA Hiroki Nakahara, Haruyoshi Yonekawa, Tomoya Fujii, Shimpei Sato Tokyo Institute of Technology,

More information

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Section 10 - Detectors part II Descriptors Mani Golparvar-Fard Department of Civil and Environmental Engineering 3129D, Newmark Civil Engineering

More information

Small is the New Big: Data Analytics on the Edge

Small is the New Big: Data Analytics on the Edge Small is the New Big: Data Analytics on the Edge An overview of processors and algorithms for deep learning techniques on the edge Dr. Abhay Samant VP Engineering, Hiller Measurements Adjunct Faculty,

More information

TEGRA K1 AND THE AUTOMOTIVE INDUSTRY. Gernot Ziegler, Timo Stich

TEGRA K1 AND THE AUTOMOTIVE INDUSTRY. Gernot Ziegler, Timo Stich TEGRA K1 AND THE AUTOMOTIVE INDUSTRY Gernot Ziegler, Timo Stich Previously: Tegra in Automotive Infotainment / Navigation Digital Instrument Cluster Passenger Entertainment TEGRA K1 with Kepler GPU GPU:

More information

Bias-Variance Trade-off (cont d) + Image Representations

Bias-Variance Trade-off (cont d) + Image Representations CS 275: Machine Learning Bias-Variance Trade-off (cont d) + Image Representations Prof. Adriana Kovashka University of Pittsburgh January 2, 26 Announcement Homework now due Feb. Generalization Training

More information

Image Matching. AKA: Image registration, the correspondence problem, Tracking,

Image Matching. AKA: Image registration, the correspondence problem, Tracking, Image Matching AKA: Image registration, the correspondence problem, Tracking, What Corresponds to What? Daisy? Daisy From: www.amphian.com Relevant for Analysis of Image Pairs (or more) Also Relevant for

More information

Exploring Task Parallelism for Heterogeneous Systems Using Multicore Task Management API

Exploring Task Parallelism for Heterogeneous Systems Using Multicore Task Management API EuroPAR 2016 ROME Workshop Exploring Task Parallelism for Heterogeneous Systems Using Multicore Task Management API Suyang Zhu 1, Sunita Chandrasekaran 2, Peng Sun 1, Barbara Chapman 1, Marcus Winter 3,

More information

Appearance-Based Place Recognition Using Whole-Image BRISK for Collaborative MultiRobot Localization

Appearance-Based Place Recognition Using Whole-Image BRISK for Collaborative MultiRobot Localization Appearance-Based Place Recognition Using Whole-Image BRISK for Collaborative MultiRobot Localization Jung H. Oh, Gyuho Eoh, and Beom H. Lee Electrical and Computer Engineering, Seoul National University,

More information

Corner Detection. GV12/3072 Image Processing.

Corner Detection. GV12/3072 Image Processing. Corner Detection 1 Last Week 2 Outline Corners and point features Moravec operator Image structure tensor Harris corner detector Sub-pixel accuracy SUSAN FAST Example descriptor: SIFT 3 Point Features

More information

SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014

SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014 SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014 SIFT SIFT: Scale Invariant Feature Transform; transform image

More information

A Hardware-Friendly Bilateral Solver for Real-Time Virtual-Reality Video

A Hardware-Friendly Bilateral Solver for Real-Time Virtual-Reality Video A Hardware-Friendly Bilateral Solver for Real-Time Virtual-Reality Video Amrita Mazumdar Armin Alaghi Jonathan T. Barron David Gallup Luis Ceze Mark Oskin Steven M. Seitz University of Washington Google

More information

Building a Panorama. Matching features. Matching with Features. How do we build a panorama? Computational Photography, 6.882

Building a Panorama. Matching features. Matching with Features. How do we build a panorama? Computational Photography, 6.882 Matching features Building a Panorama Computational Photography, 6.88 Prof. Bill Freeman April 11, 006 Image and shape descriptors: Harris corner detectors and SIFT features. Suggested readings: Mikolajczyk

More information

Large scale Imaging on Current Many- Core Platforms

Large scale Imaging on Current Many- Core Platforms Large scale Imaging on Current Many- Core Platforms SIAM Conf. on Imaging Science 2012 May 20, 2012 Dr. Harald Köstler Chair for System Simulation Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen,

More information

Parallel Tracking. Henry Spang Ethan Peters

Parallel Tracking. Henry Spang Ethan Peters Parallel Tracking Henry Spang Ethan Peters Contents Introduction HAAR Cascades Viola Jones Descriptors FREAK Descriptor Parallel Tracking GPU Detection Conclusions Questions Introduction Tracking is a

More information

PERFORMANCE MEASUREMENTS OF FEATURE TRACKING AND HISTOGRAM BASED TRAFFIC CONGESTION ALGORITHMS

PERFORMANCE MEASUREMENTS OF FEATURE TRACKING AND HISTOGRAM BASED TRAFFIC CONGESTION ALGORITHMS PERFORMANCE MEASUREMENTS OF FEATURE TRACKING AND HISTOGRAM BASED TRAFFIC CONGESTION ALGORITHMS Ozgur Altun 1 and Kenan Aksoy 2 Proline Bilisim Sistemleri, Istanbul, Turkey 1 Research and Development Engineer,

More information

Local Image Features

Local Image Features Local Image Features Computer Vision CS 143, Brown Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial This section: correspondence and alignment

More information

Local Patch Descriptors

Local Patch Descriptors Local Patch Descriptors Slides courtesy of Steve Seitz and Larry Zitnick CSE 803 1 How do we describe an image patch? How do we describe an image patch? Patches with similar content should have similar

More information

Adaptable Intelligence The Next Computing Era

Adaptable Intelligence The Next Computing Era Adaptable Intelligence The Next Computing Era Hot Chips, August 21, 2018 Victor Peng, CEO, Xilinx Pervasive Intelligence from Cloud to Edge to Endpoints >> 1 Exponential Growth and Opportunities Data Explosion

More information

Local Image Features

Local Image Features Local Image Features Computer Vision Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Flashed Face Distortion 2nd Place in the 8th Annual Best

More information

Lecture 10 Detectors and descriptors

Lecture 10 Detectors and descriptors Lecture 10 Detectors and descriptors Properties of detectors Edge detectors Harris DoG Properties of detectors SIFT Shape context Silvio Savarese Lecture 10-26-Feb-14 From the 3D to 2D & vice versa P =

More information

Features Points. Andrea Torsello DAIS Università Ca Foscari via Torino 155, Mestre (VE)

Features Points. Andrea Torsello DAIS Università Ca Foscari via Torino 155, Mestre (VE) Features Points Andrea Torsello DAIS Università Ca Foscari via Torino 155, 30172 Mestre (VE) Finding Corners Edge detectors perform poorly at corners. Corners provide repeatable points for matching, so

More information

THE LEADER IN VISUAL COMPUTING

THE LEADER IN VISUAL COMPUTING MOBILE EMBEDDED THE LEADER IN VISUAL COMPUTING 2 TAKING OUR VISION TO REALITY HPC DESIGN and VISUALIZATION AUTO GAMING 3 BEST DEVELOPER EXPERIENCE Tools for Fast Development Debug and Performance Tuning

More information

HEAD HardwarE Accelerated Deduplication

HEAD HardwarE Accelerated Deduplication HEAD HardwarE Accelerated Deduplication Final Report CS710 Computing Acceleration with FPGA December 9, 2016 Insu Jang Seikwon Kim Seonyoung Lee Executive Summary A-Z development of deduplication SW version

More information

SIFT Descriptor Extraction on the GPU for Large-Scale Video Analysis. Hannes Fassold, Jakub Rosner

SIFT Descriptor Extraction on the GPU for Large-Scale Video Analysis. Hannes Fassold, Jakub Rosner SIFT Descriptor Extraction on the GPU for Large-Scale Video Analysis Hannes Fassold, Jakub Rosner 2014-03-26 2 Overview GPU-activities @ AVM research group SIFT descriptor extraction Algorithm GPU implementation

More information

Real-Time Support for GPU. GPU Management Heechul Yun

Real-Time Support for GPU. GPU Management Heechul Yun Real-Time Support for GPU GPU Management Heechul Yun 1 This Week Topic: Real-Time Support for General Purpose Graphic Processing Unit (GPGPU) Today Background Challenges Real-Time GPU Management Frameworks

More information

Experiences Using Tegra K1 and X1 for Highly Energy Efficient Computing

Experiences Using Tegra K1 and X1 for Highly Energy Efficient Computing Experiences Using Tegra K1 and X1 for Highly Energy Efficient Computing Gaurav Mitra Andrew Haigh Luke Angove Anish Varghese Eric McCreath Alistair P. Rendell Research School of Computer Science Australian

More information

A System of Image Matching and 3D Reconstruction

A System of Image Matching and 3D Reconstruction A System of Image Matching and 3D Reconstruction CS231A Project Report 1. Introduction Xianfeng Rui Given thousands of unordered images of photos with a variety of scenes in your gallery, you will find

More information

Implementation and Comparison of Feature Detection Methods in Image Mosaicing

Implementation and Comparison of Feature Detection Methods in Image Mosaicing IOSR Journal of Electronics and Communication Engineering (IOSR-JECE) e-issn: 2278-2834,p-ISSN: 2278-8735 PP 07-11 www.iosrjournals.org Implementation and Comparison of Feature Detection Methods in Image

More information

Feature Detection and Matching

Feature Detection and Matching and Matching CS4243 Computer Vision and Pattern Recognition Leow Wee Kheng Department of Computer Science School of Computing National University of Singapore Leow Wee Kheng (CS4243) Camera Models 1 /

More information

27 March 2018 Mikael Arguedas and Morgan Quigley

27 March 2018 Mikael Arguedas and Morgan Quigley 27 March 2018 Mikael Arguedas and Morgan Quigley Separate devices: (prototypes 0-3) Unified camera: (prototypes 4-5) Unified system: (prototypes 6+) USB3 USB Host USB3 USB2 USB3 USB Host PCIe root

More information

OpenCL-Based Design of an FPGA Accelerator for Phase-Based Correspondence Matching

OpenCL-Based Design of an FPGA Accelerator for Phase-Based Correspondence Matching 90 Int'l Conf. Par. and Dist. Proc. Tech. and Appl. PDPTA'15 OpenCL-Based Design of an FPGA Accelerator for Phase-Based Correspondence Matching Shunsuke Tatsumi, Masanori Hariyama, Mamoru Miura, Koichi

More information

Target Shape Identification for Nanosatellites using Monocular Point Cloud Techniques

Target Shape Identification for Nanosatellites using Monocular Point Cloud Techniques Target Shape Identification for Nanosatellites using Monocular Point Cloud Techniques 6th European CubeSat Symposium Oct. 16, 2014 Mark A. Post and Xiu.T. Yan Space Mechatronic Systems Technology (SMeSTech)

More information

CAP 5415 Computer Vision Fall 2012

CAP 5415 Computer Vision Fall 2012 CAP 5415 Computer Vision Fall 01 Dr. Mubarak Shah Univ. of Central Florida Office 47-F HEC Lecture-5 SIFT: David Lowe, UBC SIFT - Key Point Extraction Stands for scale invariant feature transform Patented

More information

Intelligent Video Analytics for Urban Management

Intelligent Video Analytics for Urban Management Smart-I Gabriele Randelli Founder & CTO Intelligent Video Analytics for Urban Management Gabriele Randelli Founder & CTO gabriele@smart-interaction.com 1 Gabriele Randelli Founder & CTO Smart- Feel Interactive

More information

Cluster-based 3D Reconstruction of Aerial Video

Cluster-based 3D Reconstruction of Aerial Video Cluster-based 3D Reconstruction of Aerial Video Scott Sawyer (scott.sawyer@ll.mit.edu) MIT Lincoln Laboratory HPEC 12 12 September 2012 This work is sponsored by the Assistant Secretary of Defense for

More information

SoC Systeme ultra-schnell entwickeln mit Vivado und Visual System Integrator

SoC Systeme ultra-schnell entwickeln mit Vivado und Visual System Integrator SoC Systeme ultra-schnell entwickeln mit Vivado und Visual System Integrator FPGA Kongress München 2017 Martin Heimlicher Enclustra GmbH Agenda 2 What is Visual System Integrator? Introduction Platform

More information

Feature Descriptors. CS 510 Lecture #21 April 29 th, 2013

Feature Descriptors. CS 510 Lecture #21 April 29 th, 2013 Feature Descriptors CS 510 Lecture #21 April 29 th, 2013 Programming Assignment #4 Due two weeks from today Any questions? How is it going? Where are we? We have two umbrella schemes for object recognition

More information

GPU Programming with Ateji PX June 8 th Ateji All rights reserved.

GPU Programming with Ateji PX June 8 th Ateji All rights reserved. GPU Programming with Ateji PX June 8 th 2010 Ateji All rights reserved. Goals Write once, run everywhere, even on a GPU Target heterogeneous architectures from Java GPU accelerators OpenCL standard Get

More information

Local Image Features

Local Image Features Local Image Features Ali Borji UWM Many slides from James Hayes, Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Overview of Keypoint Matching 1. Find a set of distinctive key- points A 1 A 2 A 3 B 3

More information

Simplify System Complexity

Simplify System Complexity 1 2 Simplify System Complexity With the new high-performance CompactRIO controller Arun Veeramani Senior Program Manager National Instruments NI CompactRIO The Worlds Only Software Designed Controller

More information

SIFT - scale-invariant feature transform Konrad Schindler

SIFT - scale-invariant feature transform Konrad Schindler SIFT - scale-invariant feature transform Konrad Schindler Institute of Geodesy and Photogrammetry Invariant interest points Goal match points between images with very different scale, orientation, projective

More information

Near Memory Key/Value Lookup Acceleration MemSys 2017

Near Memory Key/Value Lookup Acceleration MemSys 2017 Near Key/Value Lookup Acceleration MemSys 2017 October 3, 2017 Scott Lloyd, Maya Gokhale Center for Applied Scientific Computing This work was performed under the auspices of the U.S. Department of Energy

More information

XPU A Programmable FPGA Accelerator for Diverse Workloads

XPU A Programmable FPGA Accelerator for Diverse Workloads XPU A Programmable FPGA Accelerator for Diverse Workloads Jian Ouyang, 1 (ouyangjian@baidu.com) Ephrem Wu, 2 Jing Wang, 1 Yupeng Li, 1 Hanlin Xie 1 1 Baidu, Inc. 2 Xilinx Outlines Background - FPGA for

More information

Visual Object Recognition

Visual Object Recognition Visual Object Recognition Lecture 3: Descriptors Per-Erik Forssén, docent Computer Vision Laboratory Department of Electrical Engineering Linköping University 2015 2014 Per-Erik Forssén Lecture 3: Descriptors

More information

Copyright 2016 Xilinx

Copyright 2016 Xilinx Zynq Architecture Zynq Vivado 2015.4 Version This material exempt per Department of Commerce license exception TSU Objectives After completing this module, you will be able to: Identify the basic building

More information

Motion illusion, rotating snakes

Motion illusion, rotating snakes Motion illusion, rotating snakes Local features: main components 1) Detection: Find a set of distinctive key points. 2) Description: Extract feature descriptor around each interest point as vector. x 1

More information

SURF. Lecture6: SURF and HOG. Integral Image. Feature Evaluation with Integral Image

SURF. Lecture6: SURF and HOG. Integral Image. Feature Evaluation with Integral Image SURF CSED441:Introduction to Computer Vision (2015S) Lecture6: SURF and HOG Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Speed Up Robust Features (SURF) Simplified version of SIFT Faster computation but

More information

Outline 7/2/201011/6/

Outline 7/2/201011/6/ Outline Pattern recognition in computer vision Background on the development of SIFT SIFT algorithm and some of its variations Computational considerations (SURF) Potential improvement Summary 01 2 Pattern

More information

Motion Estimation and Optical Flow Tracking

Motion Estimation and Optical Flow Tracking Image Matching Image Retrieval Object Recognition Motion Estimation and Optical Flow Tracking Example: Mosiacing (Panorama) M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003 Example 3D Reconstruction

More information

Computer Vision Algorithm Acceleration Using GPGPU and the Tegra Processor's Unified Memory

Computer Vision Algorithm Acceleration Using GPGPU and the Tegra Processor's Unified Memory Engineering, Operations & Technology Boeing Research & Technology Computer Vision Algorithm Acceleration Using GPGPU and the Tegra Processor's Unified Memory Aaron Mosher Boeing Research & Technology Avionics

More information

Dense matching GPU implementation

Dense matching GPU implementation Dense matching GPU implementation Author: Hailong Fu. Supervisor: Prof. Dr.-Ing. Norbert Haala, Dipl. -Ing. Mathias Rothermel. Universität Stuttgart 1. Introduction Correspondence problem is an important

More information

Towards smart sensing based on field-programmable technology

Towards smart sensing based on field-programmable technology Towards smart sensing based on field-programmable technology Wayne Luk Imperial College 28 January 2017 Outline 1. Smart sensing + field-programmable technology 2. Example: remote sensing 3. Acceleration:

More information

Computer Vision on Tegra K1. Chen Sagiv SagivTech Ltd.

Computer Vision on Tegra K1. Chen Sagiv SagivTech Ltd. Computer Vision on Tegra K1 Chen Sagiv SagivTech Ltd. Established in 2009 and headquartered in Israel Core domain expertise: GPU Computing and Computer Vision What we do: - Technology - Solutions - Projects

More information

8/28/12. CSE 820 Graduate Computer Architecture. Richard Enbody. Dr. Enbody. 1 st Day 2

8/28/12. CSE 820 Graduate Computer Architecture. Richard Enbody. Dr. Enbody. 1 st Day 2 CSE 820 Graduate Computer Architecture Richard Enbody Dr. Enbody 1 st Day 2 1 Why Computer Architecture? Improve coding. Knowledge to make architectural choices. Ability to understand articles about architecture.

More information

A Performance and Energy Comparison of FPGAs, GPUs, and Multicores for Sliding-Window Applications

A Performance and Energy Comparison of FPGAs, GPUs, and Multicores for Sliding-Window Applications A Performance and Energy Comparison of FPGAs, GPUs, and Multicores for Sliding-Window Applications Jeremy Fowers, Greg Brown, Patrick Cooke, Greg Stitt University of Florida Department of Electrical and

More information

Realization of CUDA-based real-time registration and target localization for high-resolution video images

Realization of CUDA-based real-time registration and target localization for high-resolution video images DOI 10.1007/s11554-016-0594-y ORIGINAL RESEARCH PAPER Realization of CUDA-based real-time registration and target localization for high-resolution video images Xiyang Zhi 1 Junhua Yan 2,3 Yiqing Hang 2

More information

GPU-based pedestrian detection for autonomous driving

GPU-based pedestrian detection for autonomous driving Procedia Computer Science Volume 80, 2016, Pages 2377 2381 ICCS 2016. The International Conference on Computational Science GPU-based pedestrian detection for autonomous driving V. Campmany 1,2, S. Silva

More information