Computational Imaging for Self-Driving Vehicles
|
|
|
- Bryce Hunt
- 5 years ago
- Views:
Transcription
1 CVPR 2018 Computational Imaging for Self-Driving Vehicles Jan Kautz Ramesh Raskar Achuta Kadambi Guy Satat
2 Computational Imaging for Self-Driving Vehicles Jan Kautz Ramesh Raskar Achuta Kadambi Guy Satat Computational Imaging Self Driving Cars Novel Sensors LIDAR Challenging Weather Open Problems Deep Learning
3 CVPR 2018 Computational Imaging for Self-Driving Vehicles Sample Slides for Module 1: Introduction to Computational Imaging and Implications for Self-Driving Cars
4 Photon Hacking Optics Displays Sensors Illumination Computational Photography Computational Light Transport Signal Processing Computer Vision Machine Learning Bit Hacking
5 Plenoptic Light Transport Adelson and Bergen The Plenoptic Function MIT Press 1991
6 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Adelson and Bergen The Plenoptic Function MIT Press 1991
7 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Wavelength Diversity (Hyperspectral Cam) Adelson and Bergen The Plenoptic Function MIT Press 1991
8 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Wavelength Diversity (Hyperspectral Cam) Polarization Diversity (Photos, Shape, Scatter) Adelson and Bergen The Plenoptic Function MIT Press 1991
9 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Wavelength Diversity (Hyperspectral Cam) Polarization Diversity (Photos, Shape, Scatter) Time of Flight (3D, Scattering) Adelson and Bergen The Plenoptic Function MIT Press 1991
10 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Bounce Index (Scattering) Wavelength Diversity (Hyperspectral Cam) Polarization Diversity (Photos, Shape, Scatter) Time of Flight (3D, Scattering) Adelson and Bergen The Plenoptic Function MIT Press 1991
11 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Bounce Index (Scattering) Wavelength Diversity (Hyperspectral Cam) Polarization Diversity (Photos, Shape, Scatter) Time of Flight (3D, Scattering) Adelson and Bergen The Plenoptic Function MIT Press 1991
12 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Bounce Index (Scattering) Wavelength Diversity (Hyperspectral Cam) Polarization Diversity (Photos, Shape, Scatter) Time of Flight (3D, Scattering) Adelson and Bergen The Plenoptic Function MIT Press 1991
13 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Bounce Index (Scattering) Wavelength Diversity (Hyperspectral Cam) Polarization Diversity (Photos, Shape, Scatter) Time of Flight (3D, Scattering) Adelson and Bergen The Plenoptic Function MIT Press 1991
14 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Bounce Index (Scattering) Wavelength Diversity (Hyperspectral Cam) Polarization Diversity (Photos, Shape, Scatter) Time of Flight (3D, Scattering) Adelson and Bergen The Plenoptic Function MIT Press 1991
15 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Bounce Index (Scattering) Wavelength Diversity (Hyperspectral Cam) Polarization Diversity (Photos, Shape, Scatter) Time of Flight (3D, Scattering) Adelson and Bergen The Plenoptic Function MIT Press 1991
16 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Bounce Index (Scattering) Wavelength Diversity (Hyperspectral Cam) Polarization Diversity (Photos, Shape, Scatter) Time of Flight (3D, Scattering) Adelson and Bergen The Plenoptic Function MIT Press 1991
17
18
19 Classification Resolution Localization Availability Any Weather Radar Ultrasonic Camera LiDAR
20
21 Visible X rays UV IR Microwave Radio Waves Wavelength Resolution Optical Contrast Non ionizing Availability of fluorophores
22 Optical Contrast Visible light X-Ray Wildlife Rehabilitation Center of Minnesota
23 Visible X rays UV IR Microwave Radio Waves Wavelength
24 Light and Matter in a Nutshell Object Lens Absorption Scattering
25 Velten 2011 Light in Flight
26 Velten et al, Siggraph 2013
27 Vetlen 2012 Seeing Around Corners
28 Raw Data
29 Single Photon Sensitive Imaging Gariepy et al. Nature Comm (2015)
30
31 Nanophotography 70 picosecond resolution [Kadambi et al 2013]
32 Real-time Localization Imaging Real Time Localization Kadambi, Zhao, Shi, Raskar. "Occluded Imaging with Time of Flight Sensors." ACM ToG 2016 (Pres. at SIGGRAPH)
33 Wavelength vs Shininess
34
35
36 5-D Transport Multi-Dimensional Light Transport
37 CVPR 2018 Computational Imaging for Autonomous Vehicles Sample Slides for Module 3: Emerging Vision Sensors for Self-Driving Cars
38 What s next for 3D imaging?
39
40 Microsoft Kinect v2
41 Microsoft Kinect v2
42 Microsoft Kinect v2
43
44 Multistripe Laser Scan
45 Multistripe Laser Scan
46 Multistripe Laser Scan NextEngine 3D $3000 USD Raster
47 Multistripe Laser Scan NextEngine 3D $3000 USD Raster
48
49 Polarized 3D 3D Photo w. $30 Pol. Filter
50 Polarized 3D 3D Photo w. $30 Pol. Filter
51 Polarized 3D 3D Photo w. $30 Pol. Filter
52 Plenoptic Light Transport Viewpoint Diversity (Light Field Cam) Bounce Index (Scattering) Wavelength Diversity (Hyperspectral Cam) Polarization Diversity (Photos, Shape, Scatter) Time of Flight (3D, Scattering) Adelson and Bergen The Plenoptic Function MIT Press 1991
53 Polarization of Light
54 Polarization of Light
55 Polarization of Light Plane of Polarization
56 Polarization of Light Plane of Polarization Plane of Polarization
57 Brewster s Angle
58 Cool 2D Photos Photo Credit: Bob Atkins
59 Cool 2D Photos Photo Credit: Bob Atkins
60 Polarization used in 2D photography But what about Polarizers for 3D Cams?
61 Shape from Polarization
62 Shape from Polarization
63 Shape from Polarization
64 Shape from Polarization
65 Shape from Polarization
66 Shape from Polarization Old Principle [Fresnel 1819] r r cos i ncos t cos i ncos t cos i ncos t cos ncos t i SfP crux: Solve for theta
67 Shape from Polarization Old Principle [Fresnel 1819] r r cos i ncos t cos i ncos t cos i ncos t cos ncos t i SfP crux: Solve for theta Need to know refractive index
68
69
70
71
72
73
74 Can use Schechner 15 ICCP
75 Image Formation Model
76 Image Formation Model Imax Imin Imax Imin I( pol) cos 2 pol 2 2
77 Image Formation Model Imax Imin Imax Imin I( pol) cos 2 pol 2 2 Suppose and '
78 Image Formation Model Imax Imin Imax Imin I( pol) cos 2 pol 2 2 Suppose and ' P Azimuthal Ambiguity problem with 2 solutions
79 Why is Shape from Polarization Unpopular? 1. \pi Ambiguity in Surface Normal
80 Why is Shape from Polarization Unpopular? 1. \pi Ambiguity in Surface Normal
81 Why is Shape from Polarization Unpopular? 1. \pi Ambiguity in Surface Normal 2. Refractive Distortion
82 Why is Shape from Polarization Unpopular? 1. \pi Ambiguity in Surface Normal 2. Refractive Distortion
83 Why is Shape from Polarization Unpopular? 1. π Ambiguity in Surface Normal 2. Refractive Distortion 3. Low SNR for some geometries 4. Usual challenges of integrating surface normals..
84 Shape from Polarization in the Lab Polarization Inverse Rendering Shape from Diffuse Polarization Miyazaki ICCV 2003 Atkinson TIP 2006
85 Shape from Polarization in the Lab Polarization Inverse Rendering Shape from Diffuse Polarization Miyazaki ICCV 2003 Atkinson TIP 2006 SfP never as popular as shading or photometric stereo
86 Frequency Analysis
87 Frequency Analysis
88 Frequency Analysis
89 Frequency Analysis
90 Frequency Analysis
91 Frequency Analysis
92 Frequency Analysis
93 Polarized 3D Fuses Depth and Polarization Spanning Tree Integration
94 Polarized 3D Fuses Depth and Polarization Spanning Tree Integration
95 Polarized 3D Fuses Depth and Polarization Spanning Tree Integration
96 Polarized 3D Fuses Depth and Polarization Spanning Tree Integration
97 Assumptions Unpolarized World Assumption Dielectric or Low-frequency Material Transition No specular interreflections Diffuse-dominant or Specular-dominant surfaces with slack
98 Challenging Materials
99 Challenging Materials Kinect
100 Challenging Materials Kinect Shading [Wu 14]
101 Challenging Materials Kinect Shading [Wu 14] Polarized 3D
102 Challenging Materials Kinect Shading [Wu 14] Polarized 3D
103 Break Lighting Assumptions Kinect
104 Break Lighting Assumptions Kinect Shading [Wu 14] Polarized 3D
105 Break Lighting Assumptions Kinect Shading [Wu 14] Polarized 3D
106 Break Lighting Assumptions Kinect Shading [Wu 14] Polarized 3D
107 Sensing with Compressive Sampling
108 Single Pixel Camera Pros and Cons Acquisition Time Single Pixel Camera FemtoPixel Camera Regular Camera Software complexity Hardware complexity
109 Lensless Imaging with a Femto-Pixel Satat, Tancik, Raskar IEEE Trans. Computational Imaging 2017
110 Lensless Imaging with a Femto-Pixel Traditional Our approach Regular pixel Femto-pixel
111 Framework for Imaging with a Femto-Pixel
112 CVPR 2018 Computational Imaging for Autonomous Vehicles Sample Slides for Module 4: Imaging in Challenging Weather Conditions
113
114
115 Light Scatters
116 How to Overcome Scattering Hardware Computational imaging Image processing
117 Lessons learned from seeing into the body
118 Optics Based Solutions Photon gating: Angle Time Polarization Not enough photons
119 Optics Based Solutions
120 Optics Based Solutions SLM Phase conjugation Long iterative process
121 Use All Photons! Computationally Invert Scattering Satat, Heshmat, Raviv, Raskar Nature Scientific Reports 2016
122
123 Estimate target Estimate scattering
124
125
126 Time
127 10,000,000,000 Slower Time
128 Scene Scatterer Measurement A t x y Sharp 2D Blurred 3D s x, y K x, y, t = m x, y, t
129 Estimating the Scattering - K x, y, t Point Spread Function Probabilistic interpretation: Probability to measure photon at specific location and time Bayes rule K x, y, t = f T t W(x, y t Probability to measure a photon at time t Given the time, probability to measure a photon at location x, y
130 Estimating the Scattering - K x, y, t K x, y, t = f T t W x, y t f T t, W x, y t Easier to estimate Assumptions: Enough samples to satisfy law of large numbers
131
132 We illuminate the entire object simultaneously with a pulse of light
133 Light scatters as it propagates through the tissue
134
135 Recovery of Slits
136 Results 5mm
137 Invariant to Layered Material Satat, Heshmat, Raskar COSI 2017
138 Properties of All Photons Imaging Recovers scatterer and target Calibration free Minimal assumptions Works with layered materials Doesn't require raster scan
139 Challenges
140 CVPR 2018 Computational Imaging for Autonomous Vehicles Sample Slides for Module 5: Data Driven Computational Imaging
Computational Imaging for Self-Driving Vehicles
CVPR 2018 Computational Imaging for Self-Driving Vehicles Jan Kautz--------Ramesh Raskar--------Achuta Kadambi--------Guy Satat Computational Imaging for Self-Driving Vehicles Jan Kautz--------Ramesh Raskar--------Achuta
Radiometry and reflectance
Radiometry and reflectance http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 16 Course announcements Homework 4 is still ongoing - Any questions?
Non-line-of-sight imaging
Non-line-of-sight imaging http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 25 Course announcements Homework 6 will be posted tonight. - Will
Overview of Active Vision Techniques
SIGGRAPH 99 Course on 3D Photography Overview of Active Vision Techniques Brian Curless University of Washington Overview Introduction Active vision techniques Imaging radar Triangulation Moire Active
Introduction to Computer Vision. Week 8, Fall 2010 Instructor: Prof. Ko Nishino
Introduction to Computer Vision Week 8, Fall 2010 Instructor: Prof. Ko Nishino Midterm Project 2 without radial distortion correction with radial distortion correction Light Light Light! How do you recover
Lecture 19: Depth Cameras. Visual Computing Systems CMU , Fall 2013
Lecture 19: Depth Cameras Visual Computing Systems Continuing theme: computational photography Cameras capture light, then extensive processing produces the desired image Today: - Capturing scene depth
Applications of Light Polarization in Vision
Applications of Light Polarization in Vision Lecture #18 Thanks to Yoav Schechner et al, Nayar et al, Larry Wolff, Ikeuchi et al Separating Reflected and Transmitted Scenes Michael Oprescu, www.photo.net
Active Light Time-of-Flight Imaging
Active Light Time-of-Flight Imaging Time-of-Flight Pulse-based time-of-flight scanners [Gvili03] NOT triangulation based short infrared laser pulse is sent from camera reflection is recorded in a very
Other Reconstruction Techniques
Other Reconstruction Techniques Ruigang Yang CS 684 CS 684 Spring 2004 1 Taxonomy of Range Sensing From Brain Curless, SIGGRAPH 00 Lecture notes CS 684 Spring 2004 2 Taxonomy of Range Scanning (cont.)
Computational Cameras: Exploiting Spatial- Angular Temporal Tradeoffs in Photography
Mitsubishi Electric Research Labs (MERL) Computational Cameras Computational Cameras: Exploiting Spatial- Angular Temporal Tradeoffs in Photography Amit Agrawal Mitsubishi Electric Research Labs (MERL)
Multi-View Stereo for Community Photo Collections Michael Goesele, et al, ICCV Venus de Milo
Vision Sensing Multi-View Stereo for Community Photo Collections Michael Goesele, et al, ICCV 2007 Venus de Milo The Digital Michelangelo Project, Stanford How to sense 3D very accurately? How to sense
Multiple View Geometry
Multiple View Geometry Martin Quinn with a lot of slides stolen from Steve Seitz and Jianbo Shi 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Our Goal The Plenoptic Function P(θ,φ,λ,t,V
3D Photography: Stereo
3D Photography: Stereo Marc Pollefeys, Torsten Sattler Spring 2016 http://www.cvg.ethz.ch/teaching/3dvision/ 3D Modeling with Depth Sensors Today s class Obtaining depth maps / range images unstructured
3D Scanning. Qixing Huang Feb. 9 th Slide Credit: Yasutaka Furukawa
3D Scanning Qixing Huang Feb. 9 th 2017 Slide Credit: Yasutaka Furukawa Geometry Reconstruction Pipeline This Lecture Depth Sensing ICP for Pair-wise Alignment Next Lecture Global Alignment Pairwise Multiple
L2 Data Acquisition. Mechanical measurement (CMM) Structured light Range images Shape from shading Other methods
L2 Data Acquisition Mechanical measurement (CMM) Structured light Range images Shape from shading Other methods 1 Coordinate Measurement Machine Touch based Slow Sparse Data Complex planning Accurate 2
TRANSIENT IMAGING. Freshman Imaging Project 2014
1 TRANSIENT IMAGING Freshman Imaging Project 2014 2 Outline What is transient imaging? Previous work: MIT, UBC Possible applications of transient imaging Theory behind transient imaging Our goals Technology
3D Photography: Active Ranging, Structured Light, ICP
3D Photography: Active Ranging, Structured Light, ICP Kalin Kolev, Marc Pollefeys Spring 2013 http://cvg.ethz.ch/teaching/2013spring/3dphoto/ Schedule (tentative) Feb 18 Feb 25 Mar 4 Mar 11 Mar 18 Mar
Introduction to Computer Vision. Introduction CMPSCI 591A/691A CMPSCI 570/670. Image Formation
Introduction CMPSCI 591A/691A CMPSCI 570/670 Image Formation Lecture Outline Light and Optics Pinhole camera model Perspective projection Thin lens model Fundamental equation Distortion: spherical & chromatic
CS6670: Computer Vision
CS6670: Computer Vision Noah Snavely Lecture 20: Light, reflectance and photometric stereo Light by Ted Adelson Readings Szeliski, 2.2, 2.3.2 Light by Ted Adelson Readings Szeliski, 2.2, 2.3.2 Properties
Rendering: Reality. Eye acts as pinhole camera. Photons from light hit objects
Basic Ray Tracing Rendering: Reality Eye acts as pinhole camera Photons from light hit objects Rendering: Reality Eye acts as pinhole camera Photons from light hit objects Rendering: Reality Eye acts as
Range Sensors (time of flight) (1)
Range Sensors (time of flight) (1) Large range distance measurement -> called range sensors Range information: key element for localization and environment modeling Ultrasonic sensors, infra-red sensors
Image-based Lighting
Image-based Lighting 10/17/15 T2 Computational Photography Derek Hoiem, University of Illinois Many slides from Debevec, some from Efros Next week Derek away for ICCV (Venice) Zhizhong and Aditya will
INFOGR Computer Graphics. J. Bikker - April-July Lecture 10: Ground Truth. Welcome!
INFOGR Computer Graphics J. Bikker - April-July 2015 - Lecture 10: Ground Truth Welcome! Today s Agenda: Limitations of Whitted-style Ray Tracing Monte Carlo Path Tracing INFOGR Lecture 10 Ground Truth
CS4670/5760: Computer Vision Kavita Bala Scott Wehrwein. Lecture 23: Photometric Stereo
CS4670/5760: Computer Vision Kavita Bala Scott Wehrwein Lecture 23: Photometric Stereo Announcements PA3 Artifact due tonight PA3 Demos Thursday Signups close at 4:30 today No lecture on Friday Last Time:
Non-line-of-sight imaging
Non-line-of-sight imaging http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 23 Course announcements Homework 6 posted, due November 30 th. -
Structured light , , Computational Photography Fall 2017, Lecture 27
Structured light http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 27 Course announcements Homework 5 has been graded. - Mean: 129. - Median:
CS5670: Computer Vision
CS5670: Computer Vision Noah Snavely Light & Perception Announcements Quiz on Tuesday Project 3 code due Monday, April 17, by 11:59pm artifact due Wednesday, April 19, by 11:59pm Can we determine shape
Understanding Variability
Understanding Variability Why so different? Light and Optics Pinhole camera model Perspective projection Thin lens model Fundamental equation Distortion: spherical & chromatic aberration, radial distortion
COMP371 COMPUTER GRAPHICS
COMP371 COMPUTER GRAPHICS SESSION 15 RAY TRACING 1 Announcements Programming Assignment 3 out today - overview @ end of the class Ray Tracing 2 Lecture Overview Review of last class Ray Tracing 3 Local
3D Shape and Indirect Appearance By Structured Light Transport
3D Shape and Indirect Appearance By Structured Light Transport CVPR 2014 - Best paper honorable mention Matthew O Toole, John Mather, Kiriakos N. Kutulakos Department of Computer Science University of
Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation
Obviously, this is a very slow process and not suitable for dynamic scenes. To speed things up, we can use a laser that projects a vertical line of light onto the scene. This laser rotates around its vertical
Extraction of surface normal and index of refraction using a pair of passive infrared polarimetric sensors
Extraction of surface normal and index of refraction using a pair of passive infrared polarimetric sensors Firooz Sadjadi Lockheed Martin Corporation Saint Paul, Minnesota firooz.sadjadi@ieee.org Farzad
3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller
3D Computer Vision Depth Cameras Prof. Didier Stricker Oliver Wasenmüller Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de
Light transport matrices
Light transport matrices http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 18 Course announcements Homework 5 has been posted. - Due on Friday
Shape from Diffuse Polarisation
Shape from Diffuse Polarisation Gary Atkinson and Edwin Hancock Department of Computer Science University of York, York, YO1 5DD, UK. atkinson@cs.york.ac.uk Abstract When unpolarised light is reflected
Sensor Modalities. Sensor modality: Different modalities:
Sensor Modalities Sensor modality: Sensors which measure same form of energy and process it in similar ways Modality refers to the raw input used by the sensors Different modalities: Sound Pressure Temperature
CS354 Computer Graphics Ray Tracing. Qixing Huang Januray 24th 2017
CS354 Computer Graphics Ray Tracing Qixing Huang Januray 24th 2017 Graphics Pipeline Elements of rendering Object Light Material Camera Geometric optics Modern theories of light treat it as both a wave
Video Georegistration: Key Challenges. Steve Blask Harris Corporation GCSD Melbourne, FL 32934
Video Georegistration: Key Challenges Steve Blask sblask@harris.com Harris Corporation GCSD Melbourne, FL 32934 Definitions Registration: image to image alignment Find pixel-to-pixel correspondences between
Multi-view stereo. Many slides adapted from S. Seitz
Multi-view stereo Many slides adapted from S. Seitz Beyond two-view stereo The third eye can be used for verification Multiple-baseline stereo Pick a reference image, and slide the corresponding window
Global Illumination. COMP 575/770 Spring 2013
Global Illumination COMP 575/770 Spring 2013 Final Exam and Projects COMP 575 Final Exam Friday, May 3 4:00 pm COMP 770 (and 575 extra credit) Projects Final report due by end of day, May 1 Presentations:
Depth. Common Classification Tasks. Example: AlexNet. Another Example: Inception. Another Example: Inception. Depth
Common Classification Tasks Recognition of individual objects/faces Analyze object-specific features (e.g., key points) Train with images from different viewing angles Recognition of object classes Analyze
Ray Optics. Lecture 23. Chapter 34. Physics II. Course website:
Lecture 23 Chapter 34 Physics II Ray Optics Course website: http://faculty.uml.edu/andriy_danylov/teaching/physicsii Today we are going to discuss: Chapter 34: Section 34.1-3 Ray Optics Ray Optics Wave
Global Illumination. Global Illumination. Direct Illumination vs. Global Illumination. Indirect Illumination. Soft Shadows.
CSCI 480 Computer Graphics Lecture 18 Global Illumination BRDFs Raytracing and Radiosity Subsurface Scattering Photon Mapping [Ch. 13.4-13.5] March 28, 2012 Jernej Barbic University of Southern California
Global Illumination. CSCI 420 Computer Graphics Lecture 18. BRDFs Raytracing and Radiosity Subsurface Scattering Photon Mapping [Ch
CSCI 420 Computer Graphics Lecture 18 Global Illumination Jernej Barbic University of Southern California BRDFs Raytracing and Radiosity Subsurface Scattering Photon Mapping [Ch. 13.4-13.5] 1 Global Illumination
Recollection. Models Pixels. Model transformation Viewport transformation Clipping Rasterization Texturing + Lights & shadows
Recollection Models Pixels Model transformation Viewport transformation Clipping Rasterization Texturing + Lights & shadows Can be computed in different stages 1 So far we came to Geometry model 3 Surface
Linearizing the Plenoptic Space
Linearizing the Plenoptic Space Grégoire Nieto1, Frédéric Devernay1, James Crowley2 LJK, Université Grenoble Alpes, France 2 LIG, Université Grenoble Alpes, France 1 1 Goal: synthesize a new view Capture/sample
3D object recognition used by team robotto
3D object recognition used by team robotto Workshop Juliane Hoebel February 1, 2016 Faculty of Computer Science, Otto-von-Guericke University Magdeburg Content 1. Introduction 2. Depth sensor 3. 3D object
10/5/09 1. d = 2. Range Sensors (time of flight) (2) Ultrasonic Sensor (time of flight, sound) (1) Ultrasonic Sensor (time of flight, sound) (2) 4.1.
Range Sensors (time of flight) (1) Range Sensors (time of flight) (2) arge range distance measurement -> called range sensors Range information: key element for localization and environment modeling Ultrasonic
Physics-based Vision: an Introduction
Physics-based Vision: an Introduction Robby Tan ANU/NICTA (Vision Science, Technology and Applications) PhD from The University of Tokyo, 2004 1 What is Physics-based? An approach that is principally concerned
3D Computer Vision 1
3D Computer Vision 1 Multiview Stereo Multiview Stereo Multiview Stereo https://www.youtube.com/watch?v=ugkb7itpnae Shape from silhouette Shape from silhouette Shape from silhouette Shape from silhouette
Representing and Computing Polarized Light in a Ray Tracer
Representing and Computing Polarized Light in a Ray Tracer A Technical Report in STS 4600 Presented to the Faculty of the School of Engineering and Applied Science University of Virginia in Partial Fulfillment
Computer Vision. Introduction
Computer Vision Introduction Filippo Bergamasco (filippo.bergamasco@unive.it) http://www.dais.unive.it/~bergamasco DAIS, Ca Foscari University of Venice Academic year 2016/2017 About this course Official
Computational light transport
Computational light transport http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 19 Course announcements Homework 5 has been posted. - Due on
Multi-view Stereo. Ivo Boyadzhiev CS7670: September 13, 2011
Multi-view Stereo Ivo Boyadzhiev CS7670: September 13, 2011 What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape
Introduction to 3D Machine Vision
Introduction to 3D Machine Vision 1 Many methods for 3D machine vision Use Triangulation (Geometry) to Determine the Depth of an Object By Different Methods: Single Line Laser Scan Stereo Triangulation
Computer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
Focal stacks and lightfields
Focal stacks and lightfields http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 11 Course announcements Homework 3 is out. - Due October 12 th.
Computer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
Phys 102 Lecture 17 Introduction to ray optics
Phys 102 Lecture 17 Introduction to ray optics 1 Physics 102 lectures on light Light as a wave Lecture 15 EM waves Lecture 16 Polarization Lecture 22 & 23 Interference & diffraction Light as a ray Lecture
Topic 9: Lighting & Reflection models. Lighting & reflection The Phong reflection model diffuse component ambient component specular component
Topic 9: Lighting & Reflection models Lighting & reflection The Phong reflection model diffuse component ambient component specular component Spot the differences Terminology Illumination The transport
Depth Sensors Kinect V2 A. Fornaser
Depth Sensors Kinect V2 A. Fornaser alberto.fornaser@unitn.it Vision Depth data It is not a 3D data, It is a map of distances Not a 3D, not a 2D it is a 2.5D or Perspective 3D Complete 3D - Tomography
Determining Reflectance Parameters and Illumination Distribution from a Sparse Set of Images for View-dependent Image Synthesis
Determining Reflectance Parameters and Illumination Distribution from a Sparse Set of Images for View-dependent Image Synthesis Ko Nishino, Zhengyou Zhang and Katsushi Ikeuchi Dept. of Info. Science, Grad.
Stereo and structured light
Stereo and structured light http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 20 Course announcements Homework 5 is still ongoing. - Make sure
3D scanning. 3D scanning is a family of technologies created as a means of automatic measurement of geometric properties of objects.
Acquiring 3D shape 3D scanning 3D scanning is a family of technologies created as a means of automatic measurement of geometric properties of objects. The produced digital model is formed by geometric
Compressive Sensing of High-Dimensional Visual Signals. Aswin C Sankaranarayanan Rice University
Compressive Sensing of High-Dimensional Visual Signals Aswin C Sankaranarayanan Rice University Interaction of light with objects Reflection Fog Volumetric scattering Human skin Sub-surface scattering
CMSC427 Shading Intro. Credit: slides from Dr. Zwicker
CMSC427 Shading Intro Credit: slides from Dr. Zwicker 2 Today Shading Introduction Radiometry & BRDFs Local shading models Light sources Shading strategies Shading Compute interaction of light with surfaces
Global Illumination. Global Illumination. Direct Illumination vs. Global Illumination. Indirect Illumination. Soft Shadows.
CSCI 420 Computer Graphics Lecture 18 Global Illumination Jernej Barbic University of Southern California BRDFs Raytracing and Radiosity Subsurface Scattering Photon Mapping [Angel Ch. 11] 1 Global Illumination
Light Transport CS434. Daniel G. Aliaga Department of Computer Science Purdue University
Light Transport CS434 Daniel G. Aliaga Department of Computer Science Purdue University Topics Local and Global Illumination Models Helmholtz Reciprocity Dual Photography/Light Transport (in Real-World)
CS4495/6495 Introduction to Computer Vision
CS4495/6495 Introduction to Computer Vision 9C-L1 3D perception Some slides by Kelsey Hawkins Motivation Why do animals, people & robots need vision? To detect and recognize objects/landmarks Is that a
The Rendering Equation. Computer Graphics CMU /15-662
The Rendering Equation Computer Graphics CMU 15-462/15-662 Review: What is radiance? Radiance at point p in direction N is radiant energy ( #hits ) per unit time, per solid angle, per unit area perpendicular
FRESNEL EQUATION RECIPROCAL POLARIZATION METHOD
FRESNEL EQUATION RECIPROCAL POLARIZATION METHOD BY DAVID MAKER, PH.D. PHOTON RESEARCH ASSOCIATES, INC. SEPTEMBER 006 Abstract The Hyperspectral H V Polarization Inverse Correlation technique incorporates
Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2010 Today Light fields Introduction Light fields Signal processing analysis Light field cameras Application Introduction Pinhole camera
CS635 Spring Department of Computer Science Purdue University
Light Transport CS635 Spring 2010 Daniel G Aliaga Daniel G. Aliaga Department of Computer Science Purdue University Topics Local and GlobalIllumination Models Helmholtz Reciprocity Dual Photography/Light
Light. Properties of light. What is light? Today What is light? How do we measure it? How does light propagate? How does light interact with matter?
Light Properties of light Today What is light? How do we measure it? How does light propagate? How does light interact with matter? by Ted Adelson Readings Andrew Glassner, Principles of Digital Image
Ligh%ng and Reflectance
Ligh%ng and Reflectance 2 3 4 Ligh%ng Ligh%ng can have a big effect on how an object looks. Modeling the effect of ligh%ng can be used for: Recogni%on par%cularly face recogni%on Shape reconstruc%on Mo%on
A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India
A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India Keshav Mahavidyalaya, University of Delhi, Delhi, India Abstract
Epipolar geometry contd.
Epipolar geometry contd. Estimating F 8-point algorithm The fundamental matrix F is defined by x' T Fx = 0 for any pair of matches x and x in two images. Let x=(u,v,1) T and x =(u,v,1) T, each match gives
specular diffuse reflection.
Lesson 8 Light and Optics The Nature of Light Properties of Light: Reflection Refraction Interference Diffraction Polarization Dispersion and Prisms Total Internal Reflection Huygens s Principle The Nature
ToF Camera for high resolution 3D images with affordable pricing
ToF Camera for high resolution 3D images with affordable pricing Basler AG Jana Bartels, Product Manager 3D Agenda Coming next I. Basler AG II. 3D Purpose and Time-of-Flight - Working Principle III. Advantages
Global Illumination CS334. Daniel G. Aliaga Department of Computer Science Purdue University
Global Illumination CS334 Daniel G. Aliaga Department of Computer Science Purdue University Recall: Lighting and Shading Light sources Point light Models an omnidirectional light source (e.g., a bulb)
Il colore: acquisizione e visualizzazione. Lezione 17: 11 Maggio 2012
Il colore: acquisizione e visualizzazione Lezione 17: 11 Maggio 2012 The importance of color information Precision vs. Perception 3D scanned geometry Photo Color and appearance Pure geometry Pure color
Using infrared proximity sensors for close 2D localization and object size recognition. Richard Berglind Neonode
Using infrared proximity sensors for close 2D localization and object size recognition Richard Berglind Neonode Outline Overview of sensor types IR proximity sensors and their drawbacks Principles of a
Mapping textures on 3D geometric model using reflectance image
Mapping textures on 3D geometric model using reflectance image Ryo Kurazume M. D. Wheeler Katsushi Ikeuchi The University of Tokyo Cyra Technologies, Inc. The University of Tokyo fkurazume,kig@cvl.iis.u-tokyo.ac.jp
Algorithms for Image-Based Rendering with an Application to Driving Simulation
Algorithms for Image-Based Rendering with an Application to Driving Simulation George Drettakis GRAPHDECO/Inria Sophia Antipolis, Université Côte d Azur http://team.inria.fr/graphdeco Graphics for Driving
Ray Optics. Lecture 23. Chapter 23. Physics II. Course website:
Lecture 23 Chapter 23 Physics II Ray Optics Course website: http://faculty.uml.edu/andriy_danylov/teaching/physicsii Let s finish talking about a diffraction grating Diffraction Grating Let s improve (more
Today. Global illumination. Shading. Interactive applications. Rendering pipeline. Computergrafik. Shading Introduction Local shading models
Computergrafik Matthias Zwicker Universität Bern Herbst 2009 Today Introduction Local shading models Light sources strategies Compute interaction of light with surfaces Requires simulation of physics Global
Acquiring 4D Light Fields of Self-Luminous Light Sources Using Programmable Filter
Acquiring 4D Light Fields of Self-Luminous Light Sources Using Programmable Filter Motohiro Nakamura 1, Takahiro Okabe 1, and Hendrik P. A. Lensch 2 1 Kyushu Institute of Technology 2 Tübingen University
CSE 167: Lecture #7: Color and Shading. Jürgen P. Schulze, Ph.D. University of California, San Diego Fall Quarter 2011
CSE 167: Introduction to Computer Graphics Lecture #7: Color and Shading Jürgen P. Schulze, Ph.D. University of California, San Diego Fall Quarter 2011 Announcements Homework project #3 due this Friday,
Computational Photography
Computational Photography Photography and Imaging Michael S. Brown Brown - 1 Part 1 Overview Photography Preliminaries Traditional Film Imaging (Camera) Part 2 General Imaging 5D Plenoptic Function (McMillan)
Phys 1020, Day 18: Questions? Cameras, Blmfld Reminders: Next Up: digital cameras finish Optics Note Final Project proposals next week!
Lights. Action. Phys 1020, Day 18: Questions? Cameras, Blmfld 15.1 Reminders: Next Up: digital cameras finish Optics Note Final Project proposals next week! 1 What have we learned in this section: 1) Lasers
Shading, lighting, & BRDF Theory. Cliff Lindsay, PHD
Shading, lighting, & BRDF Theory Cliff Lindsay, PHD Overview of today s lecture BRDF Characteristics Lights in terms of BRDFs Classes of BRDFs Ambient light & Shadows in terms of BRDFs Decomposing Reflection
More computational light transport
More computational light transport http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 23 Course announcements Sign-up for final project checkpoint
Ray Tracing. CSCI 420 Computer Graphics Lecture 15. Ray Casting Shadow Rays Reflection and Transmission [Ch ]
![Ray Tracing. CSCI 420 Computer Graphics Lecture 15. Ray Casting Shadow Rays Reflection and Transmission [Ch ]](/thumbs/78/78594982.jpg "Ray Tracing. CSCI 420 Computer Graphics Lecture 15. Ray Casting Shadow Rays Reflection and Transmission [Ch ]")
CSCI 420 Computer Graphics Lecture 15 Ray Tracing Ray Casting Shadow Rays Reflection and Transmission [Ch. 13.2-13.3] Jernej Barbic University of Southern California 1 Local Illumination Object illuminations
Image-Based Rendering
Image-Based Rendering COS 526, Fall 2016 Thomas Funkhouser Acknowledgments: Dan Aliaga, Marc Levoy, Szymon Rusinkiewicz What is Image-Based Rendering? Definition 1: the use of photographic imagery to overcome
DIFFUSE-SPECULAR SEPARATION OF MULTI-VIEW IMAGES UNDER VARYING ILLUMINATION. Department of Artificial Intelligence Kyushu Institute of Technology
DIFFUSE-SPECULAR SEPARATION OF MULTI-VIEW IMAGES UNDER VARYING ILLUMINATION Kouki Takechi Takahiro Okabe Department of Artificial Intelligence Kyushu Institute of Technology ABSTRACT Separating diffuse
The Focused Plenoptic Camera. Lightfield photographers, focus your cameras! Karl Marx
The Focused Plenoptic Camera Lightfield photographers, focus your cameras! Karl Marx Plenoptic Camera, Adelson 1992 Main lens focused on microlenses Plenoptic Camera, Adelson 1992 Microlenses focused on
3. Confirm Does the law of reflection apply to rough surfaces? Explain. Diffuse Reflection
Light Key Concepts How does light reflect from smooth surfaces and rough surfaces? What happens to light when it strikes a concave mirror? Which types of mirrors can produce a virtual image? Reflection
Passive 3D Photography
SIGGRAPH 99 Course on 3D Photography Passive 3D Photography Steve Seitz Carnegie Mellon University http:// ://www.cs.cmu.edu/~seitz Talk Outline. Visual Cues 2. Classical Vision Algorithms 3. State of
Optical Active 3D Scanning. Gianpaolo Palma
Optical Active 3D Scanning Gianpaolo Palma 3D Scanning Taxonomy SHAPE ACQUISTION CONTACT NO-CONTACT NO DESTRUCTIVE DESTRUCTIVE X-RAY MAGNETIC OPTICAL ACOUSTIC CMM ROBOTIC GANTRY SLICING ACTIVE PASSIVE
Interactive Methods in Scientific Visualization
Interactive Methods in Scientific Visualization GPU Volume Raycasting Christof Rezk-Salama University of Siegen, Germany Volume Rendering in a Nutshell Image Plane Eye Data Set Back-to-front iteration