The Computational Photometer
|
|
|
- Jordan May
- 5 years ago
- Views:
Transcription
1 The Computational Photometer Hybrid FPGA Digital Video Transformation for 3D Joint UAA and CU-Boulder Project March 28, 2014 Sam Siewert
2 The CP Team Sam Siewert PI, UAA, Adjunct CU-Boulder Vitaly Ivanov BS Student, UAA, Verification, Lab Content Jay Khandhar MS Student, CU-Boulder, DE2i & Linux Randall Meyers TME, Mentor Graphics, PCB layout Jeries Shihadeh Ph.D. Student, CU-Boulder, HW and FW Sagar Sidhpura MS Student, CU-Boulder, Linux Real-Time Kernel Sponsors: Intel Embedded Education and Research Program (DE2i) Mentor Graphics, Longmont Colorado (DxDesigner, Randall Meyer s PCB Work, Fabrication) Altera University Program (DE0, DE4) Sam Siewert 2
3 Computational Photometer Computational Photography Extension Continuous Computer and Machine Vision Processing Acceleration in Embedded Systems CV Co-Processor Between Photometer and CPU Performs Function Like a GPU, but For CV Computer Vision Turing Test - Inverse Rendering Create a World/Scene Model From Image Sequence Multi-Channel (Passive 3D, Multi-Spectral) Open Hardware Reference, Low-Cost, Real-Time Performance Sam Siewert 3
4 2D & 3D Passive Computational 3D Disparity & Depth Map Photometry 2D Skeletal Transform Networked Video Analytics Analog Camera #1 LEFT (NIR, Visible) Analog Camera #2 RIGHT (NIR, Visible) Altera FPGA CVPU (Computer Vision Processing Unit) USB 2.0, PCIe Host Channels 2D Hough Transform Flash SD Card Mobile Sensor Network Processor (TI OMAP, Atom) HD Digital Camera Port (Snapshot) Sam Siewert 4
![Simple Continuous Image Processing {P} = clamp[ {Q}*alpha + beta]](/docs-images/91/107400548/images/5-1.jpg "Brightness & Contrast Sequential Vector Instructions SSE, NEON,")
5 Simple Continuous Image Processing {P} = clamp[ {Q}*alpha + beta] Brightness & Contrast Sequential Vector Instructions SSE, NEON, Altivec Thread Grid Multi-Core Thread Grid with Vector Instructions - GPU Concurrent State Machine Co-Processor FPGA, ASIC Sam Siewert 5
6 A Few More Issues... Temporal and Spatial Locality (Memory Access) - Halide API (E.g. OpenCV) vs. Language (E.g. Halide) Hardware Acceleration GPU, DSP, Codec, FPGA Embedding and Efficiency Watts / Frame/sec DCT idct PSF Sam Siewert 6
7 Computational Photometer Goals 1. Education Reference Hardware, Firmware, Software for Students (Analog IR/Visible Photometer Interface, CameraPort HD Snapshots) 2. Innovation Product Exploration and Definition for CV Applications (Wound Care, Ice Hazards, UAV natural resource surveys, Robotics & Automation) 3. Applied Research FPGA Acceleration of Traditional Image Transforms as well as Bio-inspired CV Algorithms for 3D and Multi-spectral Applications 4. Fundamental Research Emulation, Interaction and Augmentation of Human Visual System Sam Siewert 7
8 Why Build a New Camera Interface? Cost, Open, RT Performance, Battery Power, 2+ Channel, Flexible Optics, Continuous Image Processing Configuration Cost Openness Performance Efficiency Score CP Low (3) Open HW, FW, SW (3) *RT (3) High (3) 12 Digital Camera Low (3) Proprietary HW, Open FW, SW (2) Variable (1) High (3) 9 Port 5 Analog Camera with PC Frame Grabber Low (3) Proprietary HW, Open FW, SW (2) RT (3) Low (1) 9 CameraLink 4 High (1) Proprietary HW, IP FW, Open SW (1.5) RT (3) High (3) 8.5 USB Webcam or Active Depth Mapper Low (3) Proprietary HW, FW, Open SW (1) Variable (1) High (3) 8 Ethernet CCTV 6 Medium (2) Proprietary HW, FW, Open SW (1) Predictable (2) Low (1) 6 HD and SD-SDI High (1) Proprietary HW, FW, SW (0) RT (3) Low (1) 5 Sam Siewert 8
9 CP Interface PCB Design Dual TI Video Decoders, DE0 Cyclone III or or DE2i Cyclone IV FPGA FIFO with Transform State Machines, Dual FTDI Uplink, I2C Configuration Sam Siewert 9
10 Small CP Interface PCB + System CP Custom PCB Replaces Lucite on DE0 Cyclone III FPGA (50K LEs) DE0 2 Component Inputs Dual USB 2.0 Uplinks Suitable for UAV Use Drop-in-Place Robotics & Automation Beagle CameraPort for Leopard HD Cameras Any NTSC Optics + CCD TI-OMAP + Linux TI-OMAP + BeagleJuice U S B Sam Siewert 10
11 Compare to Video Analytics Configurations (All OTS) Power Efficiency, Embedded Applications, High-End Computer & Machine Vision Performance, Open Compare to GP-GPU, Many-Core and Back-hauling DV from Wireless/OTA and Ethernet Cameras to Data Centers 1. Placing Computation at Camera Interface (Innovation) 2. Acceleration of Multi-Channel Bio-Inspired Algorithms for 3D and Multi-Spectral Active/Passive Perception (Research) 3. Enabling Student R&D (Education) Sam Siewert 11
12 3D Scene Parsing - Research Human Depth Cues (Physiology, Psychology, Physics) Between 9 and 15 Recognized Cues James Cutting, Peter Vishton Machine Vision Methods 1. Structure from Motion (Suitable for UAV, Photogrammetry Elevation Estimations) - Passive 2. Binocular Stereopsis (Two Channel Registration and Triangulation with Camera Extrinsic and Intrinsic Calibration) - Passive 3. Structured Light Projection (PrimeSense) - Active 4. Time of Flight (LIDAR) - Active Active Methods vs. Passive IEEE RAS Paper on CV Improvement with 3D Change Their Perception, December 2013 IEEE RAS Sam Siewert 12
13 Feature Vector Keypoints Continuous Feature Vector Keypoint Generation Requires Pyramidal Gaussian Resolution Decimation and Interpolation for Up-conversion (FPGA) OpenCV Image Pyramids Low Pass Filtering (Gaussian kernel convolution) followed by pixel decimation (removal of odd or even numbered rows and columns) Requires Gradient (Edge) Computations Software-based Storage and Search L=0, e.g. 9x9 L=1, 5x5 L=2, 3x3 Rows 0 8, Col 0 8 Drop rows 1,3,5,7 to go to L=1 5x5 Drop rows 1,3 to go to L=2 3x3 Sam Siewert 13
14 Image Correspondence Mosaics (Stitching) Stereopsis Recognition Structure from Motion Left-Eye Right-Eye Awareness, Recognition, Security, Safety Sam Siewert 14
15 Testing and Research Plans Enable UAA and CU-Boulder and University and Developer Programs (Open Hardware) 1. UAA Ice Formation (Multi-Spectral NIR + Visible Channels) at Port of Anchorage 2. CU-Boulder Plant Growth Chamber (NIR + Visible) 3. Wound Care (RGB-D Active, TI DLP Module with NIR + Visible) Sam Siewert 15
16 Active RGB-Depth OTS Alternatives Occipital Boulder Colorado Kickstarter (Mobile Phone RGB-D) ASUS Xtion (Prime Sense) We are testing with OpenNI Microsoft Kinect PrimeSense Developer Cameras Intel Creative Camera and Perceptual SDK We are testing all of them and comparing None are Open Hardware Many run OpenNI Middleware, but Hardware is Proprietary Sam Siewert 16
17 Research & Education Goals - Summary Education Open Hardware, Firmware and Software Analog layer, Digital, Firmware, and Linux Software Probing and Tracing at All Layers Starting Point for Capstone Design and Student Research Research Compare Passive Binocular Vision with Computational Photometry Parallelism to Active RGB-Depth Binocular = 2 Visible Coordinated Channels (UAV) RGB-D = Active Structure IR Projection, IR & Visible Channel Low-Cost Infrared + Visible Computational Photometer for Remote Sensing and Safety/Security Applications Addition of IMU-on-Chip for Proprioception (Coordinated 3D Vision and Robotic Actuation) Sam Siewert 17
18 2D & 3D Passive Computational 3D Disparity & Depth Map Photometry 2D Skeletal Transform Networked Video Analytics Analog Camera #1 LEFT (NIR, Visible) Analog Camera #2 RIGHT (NIR, Visible) Altera FPGA CVPU (Computer Vision Processing Unit) USB 2.0, PCIe Host Channels 2D Hough Transform Flash SD Card Mobile Sensor Network Processor (TI OMAP, Atom) HD Digital Camera Port (Snapshot) Sam Siewert 18
19 FPGA State Machines YCrCb FIFO for Decoder Pixel Output Basic Frame Operations on FIFO Data Time Decimation (Nth frame) Pyramid Resolution up and down conversion (SD 4:3 AR) Pixel Re-encoding, for example to RGB from YCrCb FTDI USB 2.0 Uplink from Each Channel Advanced Transformation Sobel and Canny Edge Detection Hough Transforms (Linear, Elliptical, General) SIFT/SURF (Stitching, 3D Correspondence, Recognition) HMAX (Dr. Martin Cenek at UAA) Sam Siewert 19
20 3D Active Computational Photometry Concept (Rev-A + TI Kit) TI DLP Light-crafter Kit IR Pattern Projection Photo credits and reference: Dr. Daniel Aliaga, Purdue University Analog Camera #2 (Near Infrared) Analog Camera #1 RGB (Visible) Altera FPGA CVPU (Computer Vision Processing Unit) Depth Map HD Digital Camera Port (Snapshot) USB 2.0, PCIe Host Channels Flash SD Card Networked Video Analytics Mobile Sensor Network Processor (TI OMAP, Atom) Sam Siewert 20
21 2D Computer Vision Transforms Enable Intelligent Systems with Human-like Vision, but Wider Spectrum (Visible & Infrared) Real-Time 2D Scene Parsing & Understanding (OpenCV) Canny Transform Skeletal Transform Hough Linear Transform Sam Siewert 21
22 3D is a Simple Concept, but Difficult in Practice Triangulation is Basic Concept Camera Characteristics Complicate (Intrinsics) Binocular Apparatus Errors (Extrinsics) Common Reference Point (Left/Right) Registration Far From Matching Human Capability RGB-D Scanning Becoming More Accurate, but Active Simplified Planar Triangulation With Perfect Lenses and Detectors And Simple Point Registration Sam Siewert 22
23 3D Computer Vision Transforms Long Range ( > 5 meters) Using Passive Binocular Methods Impractical to Project from a UAV or Long Range Observer Requires Image Registration Accurate Camera Intrinsic (Camera Characteristics) & Extrinsic (e.g. Baseline) Short Range ( < 5 meters), Structured IR Light Projection for RGB-D Compare to ASUS Xtion and PrimeSense Off-the-Shelf Robust Depth Maps with Less Noise Showing Significant Promise to Improve CV Scene Segmentation and Object Recognition Compared to 2D Change Their Perception, By Xiaofeng Ren, Dieter Fox, and Kurt Konolige, IEEE RAS, December Noise in Passive Depth Maps Robust Active Depth Map Change Their Perception, By Xiaofeng Ren, Dieter Fox, and Kurt Konolige, IEEE RAS, Sam Siewert December
24 The UAA Computer Engineering Prototype and Assembly Lab Supports Operating Systems (with Hardware) Computer Vision Lab DE2i, DE0, TI-OMAP Alaska Space Grant Fellowship Lab (Autonomous Submersible, Computer Vision Guided) General Computer Engineering and Capstone Sam Siewert 24
25 Breadboard Prototype Tested Summer 2013 NTSC -> TI Decoder -> Cyclone-III FPGA -> FTDI -> Beagle xm / Beagle Bone Black (Linux) Model-Sim Verification, Data-Flow Verification FTDI Interface and Linux USB Custom Driver (Stubbed) Spec to Mentor Graphics for PCB Layout, 4-Layer Fab Sam Siewert 25
26 Goal to Release in 2014 Verification In Progress PCB Layout Done Driver Stubbed in Linux for USB and UVC Working on FPGA Transforms and Applications Testing with DE2i for 150K LE Transform Engine Sam Siewert 26
27 Schedule 2013 High Level Design Schematic Capture PCB Layout Rev-A Boards Fabricated December 2013 Rev-A Board Verification Started 2014 Prototype CU-Cam / CP Linux Driver Based on UVC Ready for Testing Dataflow Tests from CU-Cam FTDI to DE2i Linux Driver (Beagle xm Driver) Rev-1.0 Computer Vision Lab Materials Complete for Internal Review and Testing with Students in CS&E A485 Spring 2014 FIFO Decimation, Resolution Down/Up-Conversion, Pixel Format Transformation Tests Stereo Vision Application Testing with OpenCV Advanced Pixel, Macroblock, Frame Transformations on DE2i Battery Power and Field Testing During Summer (Anchorage) Post Reference Designs on Intel and Mentor University Program Sites Sam Siewert 27
28 Summary Open Reference Design for Distribution by Mentor Graphics and Intel Embedded Education and Research Program Exposes Students to High Quality 3D Vision Configurable Research Platform for 3D Passive & Active Mapping and Multi-spectral Low Cost Research Platform that Can Be Battery Powered Sam Siewert 28
The Computational Photometer
The Computational Photometer Arctic Multi-Spectral and Passive 3D Imaging Project November 2014 - Status December 4, 2014 Sam Siewert The Goal Low-Cost Multi-Channel Imager Two Channel Visible for Passive
CS A485 Computer and Machine Vision
CS A485 Computer and Machine Vision Lecture 1 Introduction Part-2 January 14, 2014 Sam Siewert Biological Vision vs. Machine Vision (Why A Honey Bee is Better than HPC for CV) Humans - 100 million Photoreceptors
Software-Defined Computational Photometry
Software-Defined Computational Photometry Arctic Multi-Spectral and Passive 3D Imaging Project April 2015 - Status April 10, 2015 Sam Siewert The Goal Low-Cost Multi-Channel Imager Two Channel Visible
Graphics Corporation, 1811 Pike Road, Longmont, CO USA ABSTRACT 1. INTRODUCTION
Low-cost, high-performance and efficiency computational photometer design Sam B. Siewert* a, Jeries Shihadeh b, Randall Myers c, Jay Khandhar b, Vitaly Ivanov a a University of Alaska Anchorage, 3211 Providence
Computer and Machine Vision
Computer and Machine Vision Lecture Week 12 Part-2 Additional 3D Scene Considerations March 29, 2014 Sam Siewert Outline of Week 12 Computer Vision APIs and Languages Alternatives to C++ and OpenCV API
Computer and Machine Vision
Computer and Machine Vision Lecture Week 4 Part-2 February 5, 2014 Sam Siewert Outline of Week 4 Practical Methods for Dealing with Camera Streams, Frame by Frame and De-coding/Re-encoding for Analysis
Computer and Machine Vision
Computer and Machine Vision Lecture Week 11 Part-2 Segmentation, Camera Calibration and Feature Alignment March 28, 2014 Sam Siewert Outline of Week 11 Exam #1 Results Overview and Solutions Wrap up of
ECEN 5043 Computer and Machine Vision
ECEN 5043 Computer and Machine Vision Lecture 1 Introduction Part-1 January 14, 2014 Sam Siewert The Course An introductory course on computer vision and machine vision. Topics covered include difference
CS A490 Machine Vision and Computer Graphics
CS A490 Machine Vision and Computer Graphics Lecture 1 - Introduction August 28, 2012 Sam Siewert Sam Siewert UC Berkeley National Research University, Philosophy/Physics 1984-85 University of Notre Dame,
The Software Defined Multi-Spectral Imager Sam Siewert ERAU, CU Boulder. Abstract. Background - 1 -
The Software Defined Multi-Spectral Imager Sam Siewert ERAU, CU Boulder Abstract The availability of Internet, line-of-sight and satellite identification and surveillance information as well as low-power,
Computer and Machine Vision
Computer and Machine Vision Lecture Week 7 Part-1 (Convolution Transform Speed-up and Hough Linear Transform) February 26, 2014 Sam Siewert Outline of Week 7 Basic Convolution Transform Speed-Up Concepts
Introduction to Computer Vision
Introduction to Computer Vision Dr. Gerhard Roth COMP 4102A Winter 2015 Version 2 General Information Instructor: Adjunct Prof. Dr. Gerhard Roth gerhardroth@rogers.com read hourly gerhardroth@cmail.carleton.ca
New Sony DepthSense TM ToF Technology
ADVANCED MATERIAL HANDLING WITH New Sony DepthSense TM ToF Technology Jenson Chang Product Marketing November 7, 2018 1 3D SENSING APPLICATIONS Pick and Place Drones Collision Detection People Counting
A320 Supplemental Multi-Core Materials
A320 Supplemental Multi-Core Materials Scaling for Data-centric Computing (Overview for OS) April 18, 2013 Sam Siewert Scaling Processors and Processing Distributed Systems Networked Machines, Map Reduce
LUMS Mine Detector Project
LUMS Mine Detector Project Using visual information to control a robot (Hutchinson et al. 1996). Vision may or may not be used in the feedback loop. Visual (image based) features such as points, lines
All human beings desire to know. [...] sight, more than any other senses, gives us knowledge of things and clarifies many differences among them.
![All human beings desire to know. [...] sight, more than any other senses, gives us knowledge of things and clarifies many differences among them.](/thumbs/91/106597332.jpg "All human beings desire to know. [...] sight, more than any other senses, gives us knowledge of things and clarifies many differences among them.")
All human beings desire to know. [...] sight, more than any other senses, gives us knowledge of things and clarifies many differences among them. - Aristotle University of Texas at Arlington Introduction
System-on-Chip Architecture for Mobile Applications. Sabyasachi Dey
System-on-Chip Architecture for Mobile Applications Sabyasachi Dey Email: sabyasachi.dey@gmail.com Agenda What is Mobile Application Platform Challenges Key Architecture Focus Areas Conclusion Mobile Revolution
INTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES. Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp.
INTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp. Computer Vision in Mobile Tegra K1 It s time! AGENDA Use cases categories
Lecture 19: Depth Cameras. Visual Computing Systems CMU , Fall 2013
Lecture 19: Depth Cameras Visual Computing Systems Continuing theme: computational photography Cameras capture light, then extensive processing produces the desired image Today: - Capturing scene depth
3D Time-of-Flight Image Sensor Solutions for Mobile Devices
3D Time-of-Flight Image Sensor Solutions for Mobile Devices SEMICON Europa 2015 Imaging Conference Bernd Buxbaum 2015 pmdtechnologies gmbh c o n f i d e n t i a l Content Introduction Motivation for 3D
CSE 4392/5369. Dr. Gian Luca Mariottini, Ph.D.
University of Texas at Arlington CSE 4392/5369 Introduction to Vision Sensing Dr. Gian Luca Mariottini, Ph.D. Department of Computer Science and Engineering University of Texas at Arlington WEB : http://ranger.uta.edu/~gianluca
New Sony DepthSense TM ToF Technology
ADVANCED MATERIAL HANDLING WITH New Sony DepthSense TM ToF Technology Jenson Chang Product Marketing November 7, 2018 1 3D SENSING APPLICATIONS Pick and Place Drones Collision Detection People Counting
HIGHLY PARALLEL COMPUTING IN PHYSICS-BASED RENDERING OpenCL Raytracing Based. Thibaut PRADOS OPTIS Real-Time & Virtual Reality Manager
HIGHLY PARALLEL COMPUTING IN PHYSICS-BASED RENDERING OpenCL Raytracing Based Thibaut PRADOS OPTIS Real-Time & Virtual Reality Manager INTRODUCTION WHO WE ARE 3 Highly Parallel Computing in Physics-based
The Benefits of GPU Compute on ARM Mali GPUs
The Benefits of GPU Compute on ARM Mali GPUs Tim Hartley 1 SEMICON Europa 2014 ARM Introduction World leading semiconductor IP Founded in 1990 1060 processor licenses sold to more than 350 companies >
27 March 2018 Mikael Arguedas and Morgan Quigley
27 March 2018 Mikael Arguedas and Morgan Quigley Separate devices: (prototypes 0-3) Unified camera: (prototypes 4-5) Unified system: (prototypes 6+) USB3 USB Host USB3 USB2 USB3 USB Host PCIe root
Miniature faking. In close-up photo, the depth of field is limited.
Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg
3D Fusion of Infrared Images with Dense RGB Reconstruction from Multiple Views - with Application to Fire-fighting Robots
3D Fusion of Infrared Images with Dense RGB Reconstruction from Multiple Views - with Application to Fire-fighting Robots Yuncong Chen 1 and Will Warren 2 1 Department of Computer Science and Engineering,
High-Fidelity Augmented Reality Interactions Hrvoje Benko Researcher, MSR Redmond
High-Fidelity Augmented Reality Interactions Hrvoje Benko Researcher, MSR Redmond New generation of interfaces Instead of interacting through indirect input devices (mice and keyboard), the user is interacting
All good things must...
Lecture 17 Final Review All good things must... UW CSE vision faculty Course Grading Programming Projects (80%) Image scissors (20%) -DONE! Panoramas (20%) - DONE! Content-based image retrieval (20%) -
Computer and Machine Vision
Computer and Machine Vision Deeper Dive into MPEG Digital Video Encoding January 22, 2014 Sam Siewert Reminders CV and MV Use UNCOMPRESSED FRAMES Remote Cameras (E.g. Security) May Need to Transport Frames
Prasanna Krishnaswamy Intel Platform Architect. Imaging Systems Design for Mixed Reality Scenarios
Prasanna Krishnaswamy Platform Architect Imaging Systems Design for Mixed Reality Scenarios Agenda Mixed Reality (MR) Introduction Key Use Cases MR System Design Overview Imaging Sensors for MR Use Case
CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
CP467 Image Processing and Pattern Recognition
CP467 Image Processing and Pattern Recognition Instructor: Hongbing Fan Introduction About DIP & PR About this course Lecture 1: an overview of DIP DIP&PR show What is Digital Image? We use digital image
Computer and Machine Vision
Computer and Machine Vision Lecture Week 12 Part-1 Additional Programming Considerations March 29, 2014 Sam Siewert Outline of Week 12 Computer Vision APIs and Languages Alternatives to C++ and OpenCV
CSE A215 Assembly Language Programming for Engineers
CSE A215 Assembly Language Programming for Engineers Lecture 1 - Introduction August 28, 2012 Sam Siewert Sam Siewert UC Berkeley National Research University, Philosophy/Physics 1984-85 University of
CONTENT ENGINEERING & VISION LABORATORY. Régis Vinciguerra
CONTENT ENGINEERING & VISION LABORATORY Régis Vinciguerra regis.vinciguerra@cea.fr ALTERNATIVE ENERGIES AND ATOMIC ENERGY COMMISSION Military Applications Division (DAM) Nuclear Energy Division (DEN) Technological
Stereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman
Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure
NI Vision Platform. Radim ŠTEFAN. ni.com
NI Vision Platform Radim ŠTEFAN www./vision National Instruments Our Stability Revenue: $1.15B in 2012 Innovation: 18% re-invested to R&D Global Operations: Approximately 7,100 employees; operations in
A Comparison between Active and Passive 3D Vision Sensors: BumblebeeXB3 and Microsoft Kinect
A Comparison between Active and Passive 3D Vision Sensors: BumblebeeXB3 and Microsoft Kinect Diana Beltran and Luis Basañez Technical University of Catalonia, Barcelona, Spain {diana.beltran,luis.basanez}@upc.edu
3D Camera for a Cellular Phone. Deborah Cohen & Dani Voitsechov Supervisor : Raja Giryes 2010/11
3D Camera for a Cellular Phone Deborah Cohen & Dani Voitsechov Supervisor : Raja Giryes 2010/11 1 Contents Why 3D? Project definition and goals Projective model of a structured light system Algorithm (3
RT Digital Media Extended Lab Choose One: Work Alone Work in a Pair Extra Work Required 3 or More Students May Collaborate, but Submissions must be Un
ECEN 5033 RT Digital Media Systems Lecture 10 Extended Lab Background March 31, 2008 Sam Siewert RT Digital Media Extended Lab Choose One: Work Alone Work in a Pair Extra Work Required 3 or More Students
Embedded HW/SW Co-Development
Embedded HW/SW Co-Development It May be Driven by the Hardware Stupid! Frank Schirrmeister EDPS 2013 Monterey April 18th SPMI USB 2.0 SLIMbus RFFE LPDDR 2 LPDDR 3 emmc 4.5 UFS SD 3.0 SD 4.0 UFS Bare Metal
Complex Sensors: Cameras, Visual Sensing. The Robotics Primer (Ch. 9) ECE 497: Introduction to Mobile Robotics -Visual Sensors
Complex Sensors: Cameras, Visual Sensing The Robotics Primer (Ch. 9) Bring your laptop and robot everyday DO NOT unplug the network cables from the desktop computers or the walls Tuesday s Quiz is on Visual
Computer and Machine Vision
Computer and Machine Vision Lecture Week 10 Part-2 Skeletal Models and Face Detection March 21, 2014 Sam Siewert Outline of Week 10 Lab #4 Overview Lab #5 and #6 Extended Lab Overview SIFT and SURF High
SIFT Descriptor Extraction on the GPU for Large-Scale Video Analysis. Hannes Fassold, Jakub Rosner
SIFT Descriptor Extraction on the GPU for Large-Scale Video Analysis Hannes Fassold, Jakub Rosner 2014-03-26 2 Overview GPU-activities @ AVM research group SIFT descriptor extraction Algorithm GPU implementation
3D Scanning. Qixing Huang Feb. 9 th Slide Credit: Yasutaka Furukawa
3D Scanning Qixing Huang Feb. 9 th 2017 Slide Credit: Yasutaka Furukawa Geometry Reconstruction Pipeline This Lecture Depth Sensing ICP for Pair-wise Alignment Next Lecture Global Alignment Pairwise Multiple
CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
Teaching Parallel Programming Using Computer Vision and Image Processing Algorithms
University of Colorado (Boulder, Denver) Teaching Parallel Programming Using Computer Vision and Image Processing Algorithms Professor Dan Connors email: Dan.Connors@ucdenver.edu Department of Electrical
Computer and Machine Vision
Computer and Machine Vision Lecture Week 5 Part-2 February 13, 2014 Sam Siewert Outline of Week 5 Background on 2D and 3D Geometric Transformations Chapter 2 of CV Fundamentals of 2D Image Transformations
GTC 2013 March San Jose, CA The Smartest People. The Best Ideas. The Biggest Opportunities. Opportunities for Participation:
GTC 2013 March 18-21 San Jose, CA The Smartest People. The Best Ideas. The Biggest Opportunities. Opportunities for Participation: SPEAK - Showcase your work among the elite of graphics computing - Call
TKT-2431 SoC design. Introduction to exercises. SoC design / September 10
TKT-2431 SoC design Introduction to exercises Assistants: Exercises and the project work Juha Arvio juha.arvio@tut.fi, Otto Esko otto.esko@tut.fi In the project work, a simplified H.263 video encoder is
SE300 SWE Practices. Lecture 10 Introduction to Event- Driven Architectures. Tuesday, March 17, Sam Siewert
SE300 SWE Practices Lecture 10 Introduction to Event- Driven Architectures Tuesday, March 17, 2015 Sam Siewert Copyright {c} 2014 by the McGraw-Hill Companies, Inc. All rights Reserved. Four Common Types
Outline. ETN-FPI Training School on Plenoptic Sensing
Outline Introduction Part I: Basics of Mathematical Optimization Linear Least Squares Nonlinear Optimization Part II: Basics of Computer Vision Camera Model Multi-Camera Model Multi-Camera Calibration
A new Computer Vision Processor Chip Design for automotive ADAS CNN applications in 22nm FDSOI based on Cadence VP6 Technology
Dr.-Ing Jens Benndorf (DCT) Gregor Schewior (DCT) A new Computer Vision Processor Chip Design for automotive ADAS CNN applications in 22nm FDSOI based on Cadence VP6 Technology Tensilica Day 2017 16th
CEC 450 Real-Time Systems
CEC 450 Real-Time Systems Lecture 1 Introduction Part 2 August 27, 2018 Sam Siewert Questions on Final Projects? Examples here - http://mercury.pr.erau.edu/~siewerts/cec450/documents/video/ Creative Projects
Visual Perception Sensors
G. Glaser Visual Perception Sensors 1 / 27 MIN Faculty Department of Informatics Visual Perception Sensors Depth Determination Gerrit Glaser University of Hamburg Faculty of Mathematics, Informatics and
Non-line-of-sight imaging
Non-line-of-sight imaging http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 25 Course announcements Homework 6 will be posted tonight. - Will
Upcoming Video Standards. Madhukar Budagavi, Ph.D. DSPS R&D Center, Dallas Texas Instruments Inc.
Upcoming Video Standards Madhukar Budagavi, Ph.D. DSPS R&D Center, Dallas Texas Instruments Inc. Outline Brief history of Video Coding standards Scalable Video Coding (SVC) standard Multiview Video Coding
Computer Vision I. Dense Stereo Correspondences. Anita Sellent 1/15/16
Computer Vision I Dense Stereo Correspondences Anita Sellent Stereo Two Cameras Overlapping field of view Known transformation between cameras From disparity compute depth [ Bradski, Kaehler: Learning
ROS-Industrial Basic Developer s Training Class
ROS-Industrial Basic Developer s Training Class Southwest Research Institute 1 Session 4: More Advanced Topics (Descartes and Perception) Southwest Research Institute 2 MOVEIT! CONTINUED 3 Motion Planning
EE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
Next Generation Verification Process for Automotive and Mobile Designs with MIPI CSI-2 SM Interface
Thierry Berdah, Yafit Snir Next Generation Verification Process for Automotive and Mobile Designs with MIPI CSI-2 SM Interface Agenda Typical Verification Challenges of MIPI CSI-2 SM designs IP, Sub System
Graduate Programs 4/08
Graduate Programs CIS Graduate Program Overview PhD Programs in Imaging Science Color Science Astrophysical Sciences and Technology (joint with the Depts.of Physics And Math) MS Programs in Imaging Science
Neue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen?
Neue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen? Fraunhofer HHI 13.07.2017 1 Fraunhofer-Gesellschaft Fraunhofer is Europe s largest organization for applied research.
Depth. Common Classification Tasks. Example: AlexNet. Another Example: Inception. Another Example: Inception. Depth
Common Classification Tasks Recognition of individual objects/faces Analyze object-specific features (e.g., key points) Train with images from different viewing angles Recognition of object classes Analyze
TIOVX TI s OpenVX Implementation
TIOVX TI s OpenVX Implementation Aish Dubey Product Marketing, Automotive Processors Embedded Vision Summit, 3 May 2017 1 TI SOC platform heterogeneous cores High level processing Object detection and
Depth Camera for Mobile Devices
Depth Camera for Mobile Devices Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Stereo Cameras Structured Light Cameras Time of Flight (ToF) Camera Inferring 3D Points Given we have
Turning an Automated System into an Autonomous system using Model-Based Design Autonomous Tech Conference 2018
Turning an Automated System into an Autonomous system using Model-Based Design Autonomous Tech Conference 2018 Asaf Moses Systematics Ltd., Technical Product Manager aviasafm@systematics.co.il 1 Autonomous
Digital Image Processing COSC 6380/4393
Digital Image Processing COSC 6380/4393 Lecture 21 Nov 16 th, 2017 Pranav Mantini Ack: Shah. M Image Processing Geometric Transformation Point Operations Filtering (spatial, Frequency) Input Restoration/
Acadia II Product Guide. Low-Power, Low-Latency Video Processing for Enhanced Vision in Any Condition
Acadia II Product Guide Low-Power, Low-Latency Video Processing for Enhanced Vision in Any Condition The Acadia II SoC is a state-of-the-art solution for integrated vision processing. Built around advanced
Image processing techniques for driver assistance. Razvan Itu June 2014, Technical University Cluj-Napoca
Image processing techniques for driver assistance Razvan Itu June 2014, Technical University Cluj-Napoca Introduction Computer vision & image processing from wiki: any form of signal processing for which
Creating Affordable and Reliable Autonomous Vehicle Systems
Creating Affordable and Reliable Autonomous Vehicle Systems Shaoshan Liu shaoshan.liu@perceptin.io Autonomous Driving Localization Most crucial task of autonomous driving Solutions: GNSS but withvariations,
Embedded Software: Its Growing Influence on the Hardware world
Embedded Software: Its Growing Influence on the Hardware world ISA Vision Summit 2009, Bangalore 16 th FEB 09 V. R. Venkatesh Head, Product Engineering Services, Wipro Technologies. Wipro in Product Engineering
Laser sensors. Transmitter. Receiver. Basilio Bona ROBOTICA 03CFIOR
Mobile & Service Robotics Sensors for Robotics 3 Laser sensors Rays are transmitted and received coaxially The target is illuminated by collimated rays The receiver measures the time of flight (back and
Dense Tracking and Mapping for Autonomous Quadrocopters. Jürgen Sturm
Computer Vision Group Prof. Daniel Cremers Dense Tracking and Mapping for Autonomous Quadrocopters Jürgen Sturm Joint work with Frank Steinbrücker, Jakob Engel, Christian Kerl, Erik Bylow, and Daniel Cremers
M100 GigE Series. Multi-Camera Vision Controller. Easy cabling with PoE. Multiple inspections available thanks to 6 GigE Vision ports and 4 USB3 ports
M100 GigE Series Easy cabling with PoE Multiple inspections available thanks to 6 GigE Vision ports and 4 USB3 ports Maximized acquisition performance through 6 GigE independent channels Common features
Depth Sensors Kinect V2 A. Fornaser
Depth Sensors Kinect V2 A. Fornaser alberto.fornaser@unitn.it Vision Depth data It is not a 3D data, It is a map of distances Not a 3D, not a 2D it is a 2.5D or Perspective 3D Complete 3D - Tomography
Computer and Machine Vision
Computer and Machine Vision Lecture Week 3 Part-2 January 27, 2014 Sam Siewert Resource Scaling Processing Co-Processors GPU CUDA, OpenCL Many-Core E.g. Intel Xeon Phi MICA FPGA E.g. Altera Stratix Ideally
Industrial Vision Days 2010 C. Strampe: ATOM oder DSP? Embedded Lösungen im Vergleich
Industrial Vision Days 2010 C. Strampe: ATOM oder DSP? Embedded Lösungen im Vergleich IMAGO Technologies GmbH Foundation since 16 years in the market. 1994 Strampe Systemelektronik 1998 Strampe Systemelektronik
Research Article Motion Control of Robot by using Kinect Sensor
Research Journal of Applied Sciences, Engineering and Technology 8(11): 1384-1388, 2014 DOI:10.19026/rjaset.8.1111 ISSN: 2040-7459; e-issn: 2040-7467 2014 Maxwell Scientific Publication Corp. Submitted:
A490 Machine Vision and Computer Graphics
A490 Machine Vision and Computer Graphics Lecture Week 11 (More on Convolutions and Introduction to Boundary, Shape and Skeletal Models) November 6, 2012 Sam Siewert More on Convolution FCG Chapter 9 -
Capturing, Modeling, Rendering 3D Structures
Computer Vision Approach Capturing, Modeling, Rendering 3D Structures Calculate pixel correspondences and extract geometry Not robust Difficult to acquire illumination effects, e.g. specular highlights
VideoQ VQTS-200 Training
VideoQ VQTS-200 Training May 2011 www.videoq.com All rights reserved. All trade marks and trade names are properties of their respective owners. VQTS 200 Training Agenda VQTS Concept and Architecture (VQL
Il colore: acquisizione e visualizzazione. Lezione 17: 11 Maggio 2012
Il colore: acquisizione e visualizzazione Lezione 17: 11 Maggio 2012 The importance of color information Precision vs. Perception 3D scanned geometry Photo Color and appearance Pure geometry Pure color
OpenCV on Zynq: Accelerating 4k60 Dense Optical Flow and Stereo Vision. Kamran Khan, Product Manager, Software Acceleration and Libraries July 2017
OpenCV on Zynq: Accelerating 4k60 Dense Optical Flow and Stereo Vision Kamran Khan, Product Manager, Software Acceleration and Libraries July 2017 Agenda Why Zynq SoCs for Traditional Computer Vision Automated
Real-time image processing and object recognition for robotics applications. Adrian Stratulat
Real-time image processing and object recognition for robotics applications Adrian Stratulat What is computer vision? Computer vision is a field that includes methods for acquiring, processing, analyzing,
Electronic Travel Aids for Blind Guidance an Industry Landscape Study
Electronic Travel Aids for Blind Guidance an Industry Landscape Study Kun (Linda) Li EECS, UC Berkeley Dec 4 th, 2015 IND ENG 290 Final Presentation Visual Impairment Traditional travel aids are limited,
M100 GigE Series. Multi-Camera Vision Controller. Easy cabling with PoE. Multiple inspections available thanks to 6 GigE Vision ports and 4 USB3 ports
M100 GigE Series Easy cabling with PoE Multiple inspections available thanks to 6 GigE Vision ports and 4 USB3 ports Maximized acquisition performance through 6 GigE independent channels Common features
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting R. Maier 1,2, K. Kim 1, D. Cremers 2, J. Kautz 1, M. Nießner 2,3 Fusion Ours 1
3D Computer Vision 1
3D Computer Vision 1 Multiview Stereo Multiview Stereo Multiview Stereo https://www.youtube.com/watch?v=ugkb7itpnae Shape from silhouette Shape from silhouette Shape from silhouette Shape from silhouette
Applications of Mobile LiDAR and UAV Sourced Photogrammetry
Applications of Mobile LiDAR and UAV Sourced Photogrammetry Thomas J. Pingel and Earle W. Isibue Northern Illinois University 2017 Illinois GIS Association (ILGISA) Annual Meeting October 2-4, 2017 tpingel.org
An introduction to 3D image reconstruction and understanding concepts and ideas
Introduction to 3D image reconstruction An introduction to 3D image reconstruction and understanding concepts and ideas Samuele Carli Martin Hellmich 5 febbraio 2013 1 icsc2013 Carli S. Hellmich M. (CERN)
Computer Vision. Introduction
Computer Vision Introduction Filippo Bergamasco (filippo.bergamasco@unive.it) http://www.dais.unive.it/~bergamasco DAIS, Ca Foscari University of Venice Academic year 2016/2017 About this course Official
P H A S E O N E I N D U S T R I A L. T h e w o r l d l e a d e r i n h i g h r e s o l u t i o n i m a g i n g
P H A S E O N E I N D U S T R I A L T h e w o r l d l e a d e r i n h i g h r e s o l u t i o n i m a g i n g 1 WE ARE A WORLD LEADING PROVIDER of medium format digital imaging systems and solutions for
Camera Calibration for a Robust Omni-directional Photogrammetry System
Camera Calibration for a Robust Omni-directional Photogrammetry System Fuad Khan 1, Michael Chapman 2, Jonathan Li 3 1 Immersive Media Corporation Calgary, Alberta, Canada 2 Ryerson University Toronto,
Model-based Visual Tracking:
Technische Universität München Model-based Visual Tracking: the OpenTL framework Giorgio Panin Technische Universität München Institut für Informatik Lehrstuhl für Echtzeitsysteme und Robotik (Prof. Alois
Embedded System Design
Embedded System Design Stephen A. Edwards Columbia University Spring 2015 Spot the Computer Cars These Days... Embedded Systems: Ubiquitous Computers iphone Laser Keyboard Nikon D300 Video Watch GPS Playstation
S7348: Deep Learning in Ford's Autonomous Vehicles. Bryan Goodman Argo AI 9 May 2017
S7348: Deep Learning in Ford's Autonomous Vehicles Bryan Goodman Argo AI 9 May 2017 1 Ford s 12 Year History in Autonomous Driving Today: examples from Stereo image processing Object detection Using RNN
Digital Systems Laboratory
2012 Fall CSE140L Digital Systems Laboratory by Dr. Choon Kim CSE Department UCSD 1 Welcome to CSE140L! 2 3-way Light Controller, 2-1 MUX, Majority Detector, 7- seg Display, Binary-to- Decimal converter.
MotionCam-3D. The highest resolution and highest accuracy area based 3D camera in the world. Jan Zizka, CEO AWARD 2018
MotionCam-3D AWARD 2018 The highest resolution and highest accuracy area based 3D camera in the world Jan Zizka, CEO 3D Scanning Scanning of static scenes 3D Scanners Scanning in motion 3D Cameras Ensenso