Photometric Stereo, Shape from Shading SfS Chapter Szelisky

Size: px
Start display at page:

Download "Photometric Stereo, Shape from Shading SfS Chapter Szelisky"

Transcription

1 Photometric Stereo, Shape from Shading SfS Chapter Szelisky Guido Gerig CS 6320, Spring 2012 Credits: M. Pollefey UNC CS256, Ohad Ben-Shahar CS BGU, Wolff JUN (

2 Photometric Stereo Depth from Shading? First step: Surface Normals from Shading Second step: Re-integration of surface from Normals

3 Examples Simulated voyage over the surface of Neptune's large moon Triton

4 Credit: Ohad Ben-Shahar CS BGU

5 Shape from Shading

6 Photometric Stereo Assume: a local shading model a set of point sources that are infinitely distant a set of pictures of an object, obtained in exactly the same camera/object configuration but using different sources A Lambertian object (or the specular component has been identified and removed)

7 Setting for Photometric Stereo Multiple images with different lighting (vs binocular/geometric stereo)

8 Goal: 3D from One View and multiple Source positions Input images Usable Data Mask

9 Scene Results Needle Diagram: Surface Normals Albedo Re-lit:

10 Projection model for surface recovery - usually called a Monge patch

11 Lambertian Reflectance Map LAMBERTIAN MODEL X E = r <n,n s > =r COS q Y Z (p,q,-1) q (ps,qs,-1) COSq 1 p 1 2 q pp 2 L qq 1 p L 2 L q 2 L

12 REFLECTANCE MAP IS A VIEWER-CENTERED REPRESENTATION OF REFLECTANCE (f x, f y, -1) = (0,1,f ) x (1,0,f ) y z=f(x,y) Surface Orientation (0,1,f ) x Z (1,0,f y ) (-f x, -f y, 1) Depth X Y dy dx IMAGE PLANE Wolff, November 4,

13 REFLECTANCE MAP IS A VIEWER-CENTERED REPRESENTATION OF REFLECTANCE (-f x, -f y, 1) = (-p, -q, 1) p, q comprise a gradient or gradient space representation for local surface orientation. Reflectance map expresses the reflectance of a material directly in terms of viewer-centered representation of local surface orientation. Wolff, November 4,

14 Reflectance Map (ps=0, qs=0)

15 Reflectance Map Credit: Ohad Ben-Shahar CS BGU

16 Reflectance Map Credit: Ohad Ben-Shahar CS BGU

17 Reflectance Map

18 p q Reflectance Map (General) z x z y p s, q s isophote

19 Reflectance Map p s, q s Isophote I Given Intensity I in image, there are multiple (p,q) combinations (= surface orientations). z p x z q y Use multiple images with different light source directions.

20 Multiple Images = Multiple Maps Can isolate p, q as contour intersection I 2 I 1 (p,q) I 1 I 2

21 Example: Two Views Still not unique for certain intensity pairs.

22 Constant Albedo Solve linear equation system to calculate N.

23 Varying Albedo Solution Forsyth & Ponce: Out of shadow: For each point source, we know the source vector (by assumption). We assume we know the scaling constant of the linear camera (k). Fold the normal (N) and the reflectance (ρ(x,y)) into one vector g, and the scaling constant and source vector into another V j. In shadow: I x, y = 0

24 Multiple Images: Linear Least Squares Approach Combine albedo and normal Separate lighting parameters More than 3 images => overdetermined system : g (x,y)=v 1 i(x,y) How to calculate albedo ρ and N? g x, y = ρ x, y N(x,y) N= g g, ρ x, y = g

25 Example LLS Input Problem: Some regions in some images are in the shadow (no image intensity).

26 Dealing with Shadows (Missing Info) For each point source, we know the source vector (by assumption). We assume we know the scaling constant of the linear camera. Fold the normal and the reflectance into one vector g, and the scaling constant and source vector into another Vj Out of shadow: I j (x, y) kb(x, y) In shadow: I j (x,y) 0 kr(x, y) N(x, y)s j g(x,y) V j No partial shadow

27 Matrix Trick for Complete Shadows Matrix from Image Vector: Multiply LHS and RHS with diag matrix I 2 1 (x,y) I 2 2 (x,y).. I 2 n (x,y) I 1 (x,y) I 2 (x,y) I n (x,y) V 1 T V 2 T.. V n T g(x,y) Known Known Known Unknown Relevant elements of the left vector and the matrix are zero at points that are in shadow.

28 Obtaining Normal and Albedo Given sufficient sources, we can solve the previous equation (most likely need a least squares solution) for g(x, y). Recall that N(x, y) is the unit normal. This means that rx,y) is the magnitude of g(x, y). This yields a check If the magnitude of g(x, y) is greater than 1, there s a problem. And N(x, y) = g(x, y) / rx,y).

29 Example LLS Input

30 Example LLS Result Reflectance / albedo:

31 Recap Obtain normal / orientation, no depth Surface Orientation Depth? IMAGE PLANE

32 Goal Shape as surface with depth and normal

33 Recovering a surface from normals - 1 Recall the surface is written as (x,y, f (x, y)) This means the normal has the form: N(x,y) f 1 x f 2 x f 2 f y y 1 1 If we write the known vector g as g(x,y) g 1 (x, y) g 2 (x, y) g 3 (x, y) Then we obtain values for the partial derivatives of the surface: f x (x,y) g 1 (x, y) g 3 (x, y) f y (x, y) g 2 (x,y) g 3 (x,y)

34 Recovering a surface from normals - 2 Recall that mixed second partials are equal --- this gives us an integrability check. We must have: g 1 (x, y) g 3 (x, y) y g 2 (x, y) g 3 (x, y) x We can now recover the surface height at any point by integration along some path, e.g. f (x, y) y 0 x 0 f x (s, y)ds f y (x,t)dt c

35 Height Map from Integration How to integrate?

36 Possible Solutions Engineering approach: Path integration (Forsyth & Ponce) In general: Calculus of Variation Approaches Horn: Characteristic Strip Method Kimmel, Siddiqi, Kimia, Bruckstein: Level set method Many others.

37 Shape by Integation (Forsyth&Ponce) The partial derivative gives the change in surface height with a small step in either the x or the y direction We can get the surface by summing these changes in height along some path.

38 Simple Algorithm Forsyth & Ponce Problem: Noise and numerical (in)accuracy are added up and result in distorted surface. Solution: Choose several different integration paths, and build average height map.

39 Mathematical Property: Integrability Smooth, C2 continuous surface: Z(x, y) xy =Z(x, y) yx p y = q x check if ( p y q x )2 is small

40 SHAPE FROM SHADING (Calculus of Variations Approach) First Attempt: Minimize error in agreement with Image Irradiance Equation over the region of interest: object 2 ( I( x, y) R( p, q)) dxdy Wolff, November 4, 1998

41 SHAPE FROM SHADING (Calculus of Variations Approach) Better Attempt: Regularize the Minimization of error in agreement with Image Irradiance Equation over the region of interest: px py q x q y ( I( x, y) R( p, q)) dxdy object Wolff, November 4, 1998

42 Horn: Characteristic Strip Method Small step in x,y change in depth: Horn, Chapter11, pp New values of p,q at this new point (x,y): (r, s, t: second partial derivatives of z(x,y) w.r.t. x and y) Hessian: curv. of surface r p x s p y q x t q y

43 Horn: Characteristic Strip Method Irradiance Equation, Reflectance Map: Horn, Chapter11, pp Derivatives (chain rule): Relationship between gradient in the image and gradient in the reflectance map

44 Horn: Characteristic Strip Method 2 Equations for 3 unknowns (r,s,t): We can t continue in artibrary direction. Horn, Chapter11, pp Trick: Specially chosen direction Step in image E(x,y) parallel to gradient in R

45 Horn: Characteristic Strip Method 2 Equations for 3 unknowns (r,s,t): We can t continue in artibrary direction. Horn, Chapter11, pp Trick: Specially chosen direction Solving for new values for p,q: Step in image E(x,y) parallel to gradient in R Change in (p,q) can be computed via gradient of image

46 Horn: Characteristic Strip Method

47 Horn: Characteristic Strip Method Solution of differential equations: Curve on surface

48 Horn: Characteristic Strip Method

49 Horn: Characteristic Strip Method Horn, Chapter11, pp

50 Another Solution to SFS: Kimmel, Siddiqi, Kimia, Bruckstein pdf document x

51 Kimmel, Siddiqi, Kimia, Bruckstein

52 Kimmel, Siddiqi, Kimia, Bruckstein

53 Kimmel, Siddiqi, Kimia, Bruckstein

54 Application Area: Geography

55 Application: Braille Code pdf document

56 Mars Rover Heads to a New Crater NYT Sept 22, 2008

57 Limitations Controlled lighting environment Specular highlights? Partial shadows? Complex interrreflections? Fixed camera Moving camera? Multiple cameras? => Another approach: binocular / geometric stereo

Photometric*Stereo* October*8,*2013* Dr.*Grant*Schindler* *

Photometric*Stereo* October*8,*2013* Dr.*Grant*Schindler* * Photometric*Stereo* October*8,*2013* Dr.*Grant*Schindler* * schindler@gatech.edu* Mul@ple*Images:*Different*Ligh@ng* Mul@ple*Images:*Different*Ligh@ng* Mul@ple*Images:*Different*Ligh@ng* Religh@ng*the*Scene*

More information

Starting this chapter

Starting this chapter Computer Vision 5. Source, Shadow, Shading Department of Computer Engineering Jin-Ho Choi 05, April, 2012. 1/40 Starting this chapter The basic radiometric properties of various light sources Develop models

More information

Capturing light. Source: A. Efros

Capturing light. Source: A. Efros Capturing light Source: A. Efros Review Pinhole projection models What are vanishing points and vanishing lines? What is orthographic projection? How can we approximate orthographic projection? Lenses

More information

Photometric Stereo. Lighting and Photometric Stereo. Computer Vision I. Last lecture in a nutshell BRDF. CSE252A Lecture 7

Photometric Stereo. Lighting and Photometric Stereo. Computer Vision I. Last lecture in a nutshell BRDF. CSE252A Lecture 7 Lighting and Photometric Stereo Photometric Stereo HW will be on web later today CSE5A Lecture 7 Radiometry of thin lenses δa Last lecture in a nutshell δa δa'cosα δacos β δω = = ( z' / cosα ) ( z / cosα

More information

Radiance. Pixels measure radiance. This pixel Measures radiance along this ray

Radiance. Pixels measure radiance. This pixel Measures radiance along this ray Photometric stereo Radiance Pixels measure radiance This pixel Measures radiance along this ray Where do the rays come from? Rays from the light source reflect off a surface and reach camera Reflection:

More information

General Principles of 3D Image Analysis

General Principles of 3D Image Analysis General Principles of 3D Image Analysis high-level interpretations objects scene elements Extraction of 3D information from an image (sequence) is important for - vision in general (= scene reconstruction)

More information

Shape from shading. Surface brightness and Surface Orientation --> Reflectance map READING: Nalwa Chapter 5. BKP Horn, Chapter 10.

Shape from shading. Surface brightness and Surface Orientation --> Reflectance map READING: Nalwa Chapter 5. BKP Horn, Chapter 10. Shape from shading Surface brightness and Surface Orientation --> Reflectance map READING: Nalwa Chapter 5. BKP Horn, Chapter 10. May 2004 SFS 1 Shading produces a compelling perception of 3-D shape. One

More information

Lambertian model of reflectance I: shape from shading and photometric stereo. Ronen Basri Weizmann Institute of Science

Lambertian model of reflectance I: shape from shading and photometric stereo. Ronen Basri Weizmann Institute of Science Lambertian model of reflectance I: shape from shading and photometric stereo Ronen Basri Weizmann Institute of Science Variations due to lighting (and pose) Relief Dumitru Verdianu Flying Pregnant Woman

More information

Other approaches to obtaining 3D structure

Other approaches to obtaining 3D structure Other approaches to obtaining 3D structure Active stereo with structured light Project structured light patterns onto the object simplifies the correspondence problem Allows us to use only one camera camera

More information

Image Processing 1 (IP1) Bildverarbeitung 1

Image Processing 1 (IP1) Bildverarbeitung 1 MIN-Fakultät Fachbereich Informatik Arbeitsbereich SAV/BV (KOGS) Image Processing 1 (IP1) Bildverarbeitung 1 Lecture 20: Shape from Shading Winter Semester 2015/16 Slides: Prof. Bernd Neumann Slightly

More information

Announcements. Photometric Stereo. Shading reveals 3-D surface geometry. Photometric Stereo Rigs: One viewpoint, changing lighting

Announcements. Photometric Stereo. Shading reveals 3-D surface geometry. Photometric Stereo Rigs: One viewpoint, changing lighting Announcements Today Photometric Stereo, next lecture return to stereo Photometric Stereo Introduction to Computer Vision CSE152 Lecture 16 Shading reveals 3-D surface geometry Two shape-from-x methods

More information

Homework 4 Computer Vision CS 4731, Fall 2011 Due Date: Nov. 15, 2011 Total Points: 40

Homework 4 Computer Vision CS 4731, Fall 2011 Due Date: Nov. 15, 2011 Total Points: 40 Homework 4 Computer Vision CS 4731, Fall 2011 Due Date: Nov. 15, 2011 Total Points: 40 Note 1: Both the analytical problems and the programming assignments are due at the beginning of class on Nov 15,

More information

COMP 558 lecture 16 Nov. 8, 2010

COMP 558 lecture 16 Nov. 8, 2010 Shading The term shading typically refers to variations in irradiance along a smooth Lambertian surface. Recall that if a surface point is illuminated by parallel light source from direction l, then the

More information

3D Motion Analysis Based on 2D Point Displacements

3D Motion Analysis Based on 2D Point Displacements 3D Motion Analysis Based on 2D Point Displacements 2D displacements of points observed on an unknown moving rigid body may provide information about - the 3D structure of the points - the 3D motion parameters

More information

Massachusetts Institute of Technology. Department of Computer Science and Electrical Engineering /6.866 Machine Vision Quiz I

Massachusetts Institute of Technology. Department of Computer Science and Electrical Engineering /6.866 Machine Vision Quiz I Massachusetts Institute of Technology Department of Computer Science and Electrical Engineering 6.801/6.866 Machine Vision Quiz I Handed out: 2004 Oct. 21st Due on: 2003 Oct. 28th Problem 1: Uniform reflecting

More information

DIRECT SHAPE FROM ISOPHOTES

DIRECT SHAPE FROM ISOPHOTES DIRECT SHAPE FROM ISOPHOTES V.Dragnea, E.Angelopoulou Computer Science Department, Stevens Institute of Technology, Hoboken, NJ 07030, USA - (vdragnea, elli)@ cs.stevens.edu KEY WORDS: Isophotes, Reflectance

More information

Ligh%ng and Reflectance

Ligh%ng and Reflectance Ligh%ng and Reflectance 2 3 4 Ligh%ng Ligh%ng can have a big effect on how an object looks. Modeling the effect of ligh%ng can be used for: Recogni%on par%cularly face recogni%on Shape reconstruc%on Mo%on

More information

Photometric Stereo. Photometric Stereo. Shading reveals 3-D surface geometry BRDF. HW3 is assigned. An example of photometric stereo

Photometric Stereo. Photometric Stereo. Shading reveals 3-D surface geometry BRDF. HW3 is assigned. An example of photometric stereo Photometric Stereo Photometric Stereo HW3 is assigned Introduction to Computer Vision CSE5 Lecture 6 Shading reveals 3-D surface geometry Shape-from-shading: Use just one image to recover shape. Requires

More information

Other Reconstruction Techniques

Other Reconstruction Techniques Other Reconstruction Techniques Ruigang Yang CS 684 CS 684 Spring 2004 1 Taxonomy of Range Sensing From Brain Curless, SIGGRAPH 00 Lecture notes CS 684 Spring 2004 2 Taxonomy of Range Scanning (cont.)

More information

CS6320: 3D Computer Vision Project 3 Shape from Shading

CS6320: 3D Computer Vision Project 3 Shape from Shading CS6320: 3D Computer Vision Project 3 Shape from Shading Arthur Coste: coste.arthur@gmail.com March 2013 1 Contents 1 Introduction 3 2 Theoretical Problems 4 2.1 Bidirectional Reflectance Function............................

More information

Announcement. Photometric Stereo. Computer Vision I. Shading reveals 3-D surface geometry. Shape-from-X. CSE252A Lecture 8

Announcement. Photometric Stereo. Computer Vision I. Shading reveals 3-D surface geometry. Shape-from-X. CSE252A Lecture 8 Announcement Photometric Stereo Lecture 8 Read Chapter 2 of Forsyth & Ponce Office hours tomorrow: 3-5, CSE 4127 5-6, CSE B260A Piazza Next lecture Shape-from-X Shading reveals 3-D surface geometry Where

More information

Announcements. Hough Transform [ Patented 1962 ] Generalized Hough Transform, line fitting. Assignment 2: Due today Midterm: Thursday, May 5 in class

Announcements. Hough Transform [ Patented 1962 ] Generalized Hough Transform, line fitting. Assignment 2: Due today Midterm: Thursday, May 5 in class Announcements Generalized Hough Transform, line fitting Assignment 2: Due today Midterm: Thursday, May 5 in class Introduction to Computer Vision CSE 152 Lecture 11a What is region like if: 1. λ 1 = 0?

More information

Comment on Numerical shape from shading and occluding boundaries

Comment on Numerical shape from shading and occluding boundaries Artificial Intelligence 59 (1993) 89-94 Elsevier 89 ARTINT 1001 Comment on Numerical shape from shading and occluding boundaries K. Ikeuchi School of Compurer Science. Carnegie Mellon dniversity. Pirrsburgh.

More information

Photometric Stereo.

Photometric Stereo. Photometric Stereo Photometric Stereo v.s.. Structure from Shading [1] Photometric stereo is a technique in computer vision for estimating the surface normals of objects by observing that object under

More information

Epipolar geometry contd.

Epipolar geometry contd. Epipolar geometry contd. Estimating F 8-point algorithm The fundamental matrix F is defined by x' T Fx = 0 for any pair of matches x and x in two images. Let x=(u,v,1) T and x =(u,v,1) T, each match gives

More information

CS6670: Computer Vision

CS6670: Computer Vision CS6670: Computer Vision Noah Snavely Lecture 20: Light, reflectance and photometric stereo Light by Ted Adelson Readings Szeliski, 2.2, 2.3.2 Light by Ted Adelson Readings Szeliski, 2.2, 2.3.2 Properties

More information

Assignment #2. (Due date: 11/6/2012)

Assignment #2. (Due date: 11/6/2012) Computer Vision I CSE 252a, Fall 2012 David Kriegman Assignment #2 (Due date: 11/6/2012) Name: Student ID: Email: Problem 1 [1 pts] Calculate the number of steradians contained in a spherical wedge with

More information

CS4670/5760: Computer Vision Kavita Bala Scott Wehrwein. Lecture 23: Photometric Stereo

CS4670/5760: Computer Vision Kavita Bala Scott Wehrwein. Lecture 23: Photometric Stereo CS4670/5760: Computer Vision Kavita Bala Scott Wehrwein Lecture 23: Photometric Stereo Announcements PA3 Artifact due tonight PA3 Demos Thursday Signups close at 4:30 today No lecture on Friday Last Time:

More information

Announcement. Lighting and Photometric Stereo. Computer Vision I. Surface Reflectance Models. Lambertian (Diffuse) Surface.

Announcement. Lighting and Photometric Stereo. Computer Vision I. Surface Reflectance Models. Lambertian (Diffuse) Surface. Lighting and Photometric Stereo CSE252A Lecture 7 Announcement Read Chapter 2 of Forsyth & Ponce Might find section 12.1.3 of Forsyth & Ponce useful. HW Problem Emitted radiance in direction f r for incident

More information

Math 113 Calculus III Final Exam Practice Problems Spring 2003

Math 113 Calculus III Final Exam Practice Problems Spring 2003 Math 113 Calculus III Final Exam Practice Problems Spring 23 1. Let g(x, y, z) = 2x 2 + y 2 + 4z 2. (a) Describe the shapes of the level surfaces of g. (b) In three different graphs, sketch the three cross

More information

Lecture 24: More on Reflectance CAP 5415

Lecture 24: More on Reflectance CAP 5415 Lecture 24: More on Reflectance CAP 5415 Recovering Shape We ve talked about photometric stereo, where we assumed that a surface was diffuse Could calculate surface normals and albedo What if the surface

More information

Multiple View Geometry

Multiple View Geometry Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric

More information

3D Shape Recovery of Smooth Surfaces: Dropping the Fixed Viewpoint Assumption

3D Shape Recovery of Smooth Surfaces: Dropping the Fixed Viewpoint Assumption IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL., NO., 1 3D Shape Recovery of Smooth Surfaces: Dropping the Fixed Viewpoint Assumption Yael Moses Member, IEEE and Ilan Shimshoni Member,

More information

Introduction to Computer Vision. Week 8, Fall 2010 Instructor: Prof. Ko Nishino

Introduction to Computer Vision. Week 8, Fall 2010 Instructor: Prof. Ko Nishino Introduction to Computer Vision Week 8, Fall 2010 Instructor: Prof. Ko Nishino Midterm Project 2 without radial distortion correction with radial distortion correction Light Light Light! How do you recover

More information

EE Light & Image Formation

EE Light & Image Formation EE 576 - Light & Electric Electronic Engineering Bogazici University January 29, 2018 EE 576 - Light & EE 576 - Light & The image of a three-dimensional object depends on: 1. Shape 2. Reflectance properties

More information

Height from Gradient with Surface Curvature and Area Constraints

Height from Gradient with Surface Curvature and Area Constraints Computer Science Department of The University of Auckland CITR at Tamaki Campus (http://www.citr.auckland.ac.nz/) CITR-TR-109 December 2001 Height from Gradient with Surface Curvature and Area Constraints

More information

Using a Raster Display Device for Photometric Stereo

Using a Raster Display Device for Photometric Stereo DEPARTMEN T OF COMP UTING SC IENC E Using a Raster Display Device for Photometric Stereo Nathan Funk & Yee-Hong Yang CRV 2007 May 30, 2007 Overview 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1. Background

More information

Reconstruction of Discrete Surfaces from Shading Images by Propagation of Geometric Features

Reconstruction of Discrete Surfaces from Shading Images by Propagation of Geometric Features Reconstruction of Discrete Surfaces from Shading Images by Propagation of Geometric Features Achille Braquelaire and Bertrand Kerautret LaBRI, Laboratoire Bordelais de Recherche en Informatique UMR 58,

More information

A Survey of Light Source Detection Methods

A Survey of Light Source Detection Methods A Survey of Light Source Detection Methods Nathan Funk University of Alberta Mini-Project for CMPUT 603 November 30, 2003 Abstract This paper provides an overview of the most prominent techniques for light

More information

Structure from motion

Structure from motion Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera

More information

Recap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?

Recap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views? Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple

More information

Image Formation I Chapter 2 (R. Szelisky)

Image Formation I Chapter 2 (R. Szelisky) Image Formation I Chapter 2 (R. Selisky) Guido Gerig CS 632 Spring 22 cknowledgements: Slides used from Prof. Trevor Darrell, (http://www.eecs.berkeley.edu/~trevor/cs28.html) Some slides modified from

More information

Lecture 22: Basic Image Formation CAP 5415

Lecture 22: Basic Image Formation CAP 5415 Lecture 22: Basic Image Formation CAP 5415 Today We've talked about the geometry of scenes and how that affects the image We haven't talked about light yet Today, we will talk about image formation and

More information

A Shape from Shading Approach for the Reconstruction of Polyhedral Objects using Genetic Algorithm

A Shape from Shading Approach for the Reconstruction of Polyhedral Objects using Genetic Algorithm A Shape from Shading Approach for the Reconstruction of Polyhedral Objects using Genetic Algorithm MANOJ KUMAR RAMA BHARGAVA R. BALASUBRAMANIAN Indian Institute of Technology Roorkee, Roorkee P.O. Box

More information

COMPUTER VISION > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE

COMPUTER VISION > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE COMPUTER VISION 2017-2018 > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE OUTLINE Optical flow Lucas-Kanade Horn-Schunck Applications of optical flow Optical flow tracking Histograms of oriented flow Assignment

More information

What is Computer Vision? Introduction. We all make mistakes. Why is this hard? What was happening. What do you see? Intro Computer Vision

What is Computer Vision? Introduction. We all make mistakes. Why is this hard? What was happening. What do you see? Intro Computer Vision What is Computer Vision? Trucco and Verri (Text): Computing properties of the 3-D world from one or more digital images Introduction Introduction to Computer Vision CSE 152 Lecture 1 Sockman and Shapiro:

More information

Understanding Variability

Understanding Variability Understanding Variability Why so different? Light and Optics Pinhole camera model Perspective projection Thin lens model Fundamental equation Distortion: spherical & chromatic aberration, radial distortion

More information

Shading and Recognition OR The first Mrs Rochester. D.A. Forsyth, UIUC

Shading and Recognition OR The first Mrs Rochester. D.A. Forsyth, UIUC Shading and Recognition OR The first Mrs Rochester D.A. Forsyth, UIUC Structure Argument: History why shading why shading analysis died reasons for hope Classical SFS+Critiques Primitives Reconstructions

More information

Announcements. Introduction. Why is this hard? What is Computer Vision? We all make mistakes. What do you see? Class Web Page is up:

Announcements. Introduction. Why is this hard? What is Computer Vision? We all make mistakes. What do you see? Class Web Page is up: Announcements Introduction Computer Vision I CSE 252A Lecture 1 Class Web Page is up: http://www.cs.ucsd.edu/classes/wi05/cse252a/ Assignment 0: Getting Started with Matlab is posted to web page, due 1/13/04

More information

L2 Data Acquisition. Mechanical measurement (CMM) Structured light Range images Shape from shading Other methods

L2 Data Acquisition. Mechanical measurement (CMM) Structured light Range images Shape from shading Other methods L2 Data Acquisition Mechanical measurement (CMM) Structured light Range images Shape from shading Other methods 1 Coordinate Measurement Machine Touch based Slow Sparse Data Complex planning Accurate 2

More information

CS 4495 Computer Vision Motion and Optic Flow

CS 4495 Computer Vision Motion and Optic Flow CS 4495 Computer Vision Aaron Bobick School of Interactive Computing Administrivia PS4 is out, due Sunday Oct 27 th. All relevant lectures posted Details about Problem Set: You may *not* use built in Harris

More information

Chapter 5 Partial Differentiation

Chapter 5 Partial Differentiation Chapter 5 Partial Differentiation For functions of one variable, y = f (x), the rate of change of the dependent variable can dy be found unambiguously by differentiation: f x. In this chapter we explore

More information

Computer Vision Lecture 17

Computer Vision Lecture 17 Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester

More information

Computer Vision Lecture 17

Computer Vision Lecture 17 Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week

More information

Image Formation I Chapter 1 (Forsyth&Ponce) Cameras

Image Formation I Chapter 1 (Forsyth&Ponce) Cameras Image Formation I Chapter 1 (Forsyth&Ponce) Cameras Guido Gerig CS 632 Spring 213 cknowledgements: Slides used from Prof. Trevor Darrell, (http://www.eecs.berkeley.edu/~trevor/cs28.html) Some slides modified

More information

Image Formation I Chapter 1 (Forsyth&Ponce) Cameras

Image Formation I Chapter 1 (Forsyth&Ponce) Cameras Image Formation I Chapter 1 (Forsyth&Ponce) Cameras Guido Gerig CS 632 Spring 215 cknowledgements: Slides used from Prof. Trevor Darrell, (http://www.eecs.berkeley.edu/~trevor/cs28.html) Some slides modified

More information

HW2. October 24, CSE 252A Computer Vision I Fall Assignment 2

HW2. October 24, CSE 252A Computer Vision I Fall Assignment 2 HW2 October 24, 2018 1 CSE 252A Computer Vision I Fall 2018 - Assignment 2 1.0.1 Instructor: David Kriegman 1.0.2 Assignment Published On: Wednesday, October 24, 2018 1.0.3 Due On: Wednesday, November

More information

> n > L > L. > h. > t. Cone by normal vector > Level curve. Base plane of object. Monge cone. image plane.

> n > L > L. > h. > t. Cone by normal vector > Level curve. Base plane of object. Monge cone. image plane. Analytical Solution of Shape from Shading Problem Seong Ik CHO, Hideo SAITO, and Shinji OAWA Department of Electrical Engineering, Keio University, Yokohama-shi, 3, Japan fsicho,saito,ozawag@ozawa.elec.keio.ac.jp

More information

Stereo: Disparity and Matching

Stereo: Disparity and Matching CS 4495 Computer Vision Aaron Bobick School of Interactive Computing Administrivia PS2 is out. But I was late. So we pushed the due date to Wed Sept 24 th, 11:55pm. There is still *no* grace period. To

More information

Photometric stereo. Recovering the surface f(x,y) Three Source Photometric stereo: Step1. Reflectance Map of Lambertian Surface

Photometric stereo. Recovering the surface f(x,y) Three Source Photometric stereo: Step1. Reflectance Map of Lambertian Surface Photometric stereo Illumination Cones and Uncalibrated Photometric Stereo Single viewpoint, multiple images under different lighting. 1. Arbitrary known BRDF, known lighting 2. Lambertian BRDF, known lighting

More information

Multivariate Calculus: Review Problems for Examination Two

Multivariate Calculus: Review Problems for Examination Two Multivariate Calculus: Review Problems for Examination Two Note: Exam Two is on Tuesday, August 16. The coverage is multivariate differential calculus and double integration. You should review the double

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem

More information

Visual Tracking (1) Feature Point Tracking and Block Matching

Visual Tracking (1) Feature Point Tracking and Block Matching Intelligent Control Systems Visual Tracking (1) Feature Point Tracking and Block Matching Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/

More information

x 6 + λ 2 x 6 = for the curve y = 1 2 x3 gives f(1, 1 2 ) = λ actually has another solution besides λ = 1 2 = However, the equation λ

x 6 + λ 2 x 6 = for the curve y = 1 2 x3 gives f(1, 1 2 ) = λ actually has another solution besides λ = 1 2 = However, the equation λ Math 0 Prelim I Solutions Spring 010 1. Let f(x, y) = x3 y for (x, y) (0, 0). x 6 + y (4 pts) (a) Show that the cubic curves y = x 3 are level curves of the function f. Solution. Substituting y = x 3 in

More information

CS 4495/7495 Computer Vision Frank Dellaert, Fall 07. Dense Stereo Some Slides by Forsyth & Ponce, Jim Rehg, Sing Bing Kang

CS 4495/7495 Computer Vision Frank Dellaert, Fall 07. Dense Stereo Some Slides by Forsyth & Ponce, Jim Rehg, Sing Bing Kang CS 4495/7495 Computer Vision Frank Dellaert, Fall 07 Dense Stereo Some Slides by Forsyth & Ponce, Jim Rehg, Sing Bing Kang Etymology Stereo comes from the Greek word for solid (στερεο), and the term can

More information

Lecture 6: Edge Detection

Lecture 6: Edge Detection #1 Lecture 6: Edge Detection Saad J Bedros sbedros@umn.edu Review From Last Lecture Options for Image Representation Introduced the concept of different representation or transformation Fourier Transform

More information

Announcements. Recognition I. Gradient Space (p,q) What is the reflectance map?

Announcements. Recognition I. Gradient Space (p,q) What is the reflectance map? Announcements I HW 3 due 12 noon, tomorrow. HW 4 to be posted soon recognition Lecture plan recognition for next two lectures, then video and motion. Introduction to Computer Vision CSE 152 Lecture 17

More information

Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers

Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers In this section we present Lagrange s method for maximizing or minimizing a general function f(x, y, z) subject to a constraint (or side condition) of the form g(x, y, z) = k. Figure 1 shows this curve

More information

3D Computer Vision Introduction

3D Computer Vision Introduction 3D Computer Vision Introduction Tom Henderson CS 6320 S2014 tch@cs.utah.edu Acknowledgements: slides from Guido Gerig (Utah) & Marc Pollefeys, UNC Chapel Hill) Administration Classes: M & W, 1:25-2:45

More information

Edge Detection Lecture 03 Computer Vision

Edge Detection Lecture 03 Computer Vision Edge Detection Lecture 3 Computer Vision Suggested readings Chapter 5 Linda G. Shapiro and George Stockman, Computer Vision, Upper Saddle River, NJ, Prentice Hall,. Chapter David A. Forsyth and Jean Ponce,

More information

PA2 Introduction to Tracking. Connected Components. Moving Object Detection. Pixel Grouping. After Pixel Grouping 2/19/17. Any questions?

PA2 Introduction to Tracking. Connected Components. Moving Object Detection. Pixel Grouping. After Pixel Grouping 2/19/17. Any questions? /19/17 PA Introduction to Tracking Any questions? Yes, its due Monday. CS 510 Lecture 1 February 15, 017 Moving Object Detection Assuming a still camera Two algorithms: Mixture of Gaussians (Stauffer Grimson)

More information

Direct Surface Reconstruction using Perspective Shape from Shading via Photometric Stereo

Direct Surface Reconstruction using Perspective Shape from Shading via Photometric Stereo Direct Surface Reconstruction using Perspective Shape from Shading via Roberto Mecca joint work with Ariel Tankus and Alfred M. Bruckstein Technion - Israel Institute of Technology Department of Computer

More information

Passive 3D Photography

Passive 3D Photography SIGGRAPH 99 Course on 3D Photography Passive 3D Photography Steve Seitz Carnegie Mellon University http:// ://www.cs.cmu.edu/~seitz Talk Outline. Visual Cues 2. Classical Vision Algorithms 3. State of

More information

Depth. Common Classification Tasks. Example: AlexNet. Another Example: Inception. Another Example: Inception. Depth

Depth. Common Classification Tasks. Example: AlexNet. Another Example: Inception. Another Example: Inception. Depth Common Classification Tasks Recognition of individual objects/faces Analyze object-specific features (e.g., key points) Train with images from different viewing angles Recognition of object classes Analyze

More information

BIL Computer Vision Apr 16, 2014

BIL Computer Vision Apr 16, 2014 BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm

More information

Subpixel Corner Detection Using Spatial Moment 1)

Subpixel Corner Detection Using Spatial Moment 1) Vol.31, No.5 ACTA AUTOMATICA SINICA September, 25 Subpixel Corner Detection Using Spatial Moment 1) WANG She-Yang SONG Shen-Min QIANG Wen-Yi CHEN Xing-Lin (Department of Control Engineering, Harbin Institute

More information

Gradient and Directional Derivatives

Gradient and Directional Derivatives Gradient and Directional Derivatives MATH 311, Calculus III J. Robert Buchanan Department of Mathematics Fall 2011 Background Given z = f (x, y) we understand that f : gives the rate of change of z in

More information

CS201: Computer Vision Introduction to Tracking

CS201: Computer Vision Introduction to Tracking CS201: Computer Vision Introduction to Tracking John Magee 18 November 2014 Slides courtesy of: Diane H. Theriault Question of the Day How can we represent and use motion in images? 1 What is Motion? Change

More information

Lecture 20: Tracking. Tuesday, Nov 27

Lecture 20: Tracking. Tuesday, Nov 27 Lecture 20: Tracking Tuesday, Nov 27 Paper reviews Thorough summary in your own words Main contribution Strengths? Weaknesses? How convincing are the experiments? Suggestions to improve them? Extensions?

More information

MIT Monte-Carlo Ray Tracing. MIT EECS 6.837, Cutler and Durand 1

MIT Monte-Carlo Ray Tracing. MIT EECS 6.837, Cutler and Durand 1 MIT 6.837 Monte-Carlo Ray Tracing MIT EECS 6.837, Cutler and Durand 1 Schedule Review Session: Tuesday November 18 th, 7:30 pm bring lots of questions! Quiz 2: Thursday November 20 th, in class (one weeks

More information

B. Examples Set up the integral(s) needed to find the area of the region bounded by

B. Examples Set up the integral(s) needed to find the area of the region bounded by Math 176 Calculus Sec. 6.1: Area Between Curves I. Area between the Curve and the x Axis A. Let f(x) 0 be continuous on [a,b]. The area of the region between the graph of f and the x-axis is A = f ( x)

More information

Orthogonal Projection Matrices. Angel and Shreiner: Interactive Computer Graphics 7E Addison-Wesley 2015

Orthogonal Projection Matrices. Angel and Shreiner: Interactive Computer Graphics 7E Addison-Wesley 2015 Orthogonal Projection Matrices 1 Objectives Derive the projection matrices used for standard orthogonal projections Introduce oblique projections Introduce projection normalization 2 Normalization Rather

More information

What you will learn today

What you will learn today What you will learn today Tangent Planes and Linear Approximation and the Gradient Vector Vector Functions 1/21 Recall in one-variable calculus, as we zoom in toward a point on a curve, the graph becomes

More information

Finally: Motion and tracking. Motion 4/20/2011. CS 376 Lecture 24 Motion 1. Video. Uses of motion. Motion parallax. Motion field

Finally: Motion and tracking. Motion 4/20/2011. CS 376 Lecture 24 Motion 1. Video. Uses of motion. Motion parallax. Motion field Finally: Motion and tracking Tracking objects, video analysis, low level motion Motion Wed, April 20 Kristen Grauman UT-Austin Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys, and S. Lazebnik

More information

Project 4 Results. Representation. Data. Learning. Zachary, Hung-I, Paul, Emanuel. SIFT and HoG are popular and successful.

Project 4 Results. Representation. Data. Learning. Zachary, Hung-I, Paul, Emanuel. SIFT and HoG are popular and successful. Project 4 Results Representation SIFT and HoG are popular and successful. Data Hugely varying results from hard mining. Learning Non-linear classifier usually better. Zachary, Hung-I, Paul, Emanuel Project

More information

CS4670/5760: Computer Vision

CS4670/5760: Computer Vision CS4670/5760: Computer Vision Kavita Bala! Lecture 28: Photometric Stereo Thanks to ScoC Wehrwein Announcements PA 3 due at 1pm on Monday PA 4 out on Monday HW 2 out on weekend Next week: MVS, sfm Last

More information

7/28/2011 SECOND HOURLY PRACTICE V Maths 21a, O.Knill, Summer 2011

7/28/2011 SECOND HOURLY PRACTICE V Maths 21a, O.Knill, Summer 2011 7/28/2011 SECOND HOURLY PRACTICE V Maths 21a, O.Knill, Summer 2011 Name: Start by printing your name in the above box. Try to answer each question on the same page as the question is asked. If needed,

More information

Schedule. MIT Monte-Carlo Ray Tracing. Radiosity. Review of last week? Limitations of radiosity. Radiosity

Schedule. MIT Monte-Carlo Ray Tracing. Radiosity. Review of last week? Limitations of radiosity. Radiosity Schedule Review Session: Tuesday November 18 th, 7:30 pm, Room 2-136 bring lots of questions! MIT 6.837 Monte-Carlo Ray Tracing Quiz 2: Thursday November 20 th, in class (one weeks from today) MIT EECS

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction

More information

Exact equations are first order DEs of the form M(x, y) + N(x, y) y' = 0 for which we can find a function f(x, φ(x)) so that

Exact equations are first order DEs of the form M(x, y) + N(x, y) y' = 0 for which we can find a function f(x, φ(x)) so that Section 2.6 Exact Equations (ONLY) Key Terms: Exact equations are first order DEs of the form M(x, y) + N(x, y) y' = 0 for which we can find a function f(x, φ(x)) so that The construction of f(x, φ(x))

More information

Stereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman

Stereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure

More information

EE795: Computer Vision and Intelligent Systems

EE795: Computer Vision and Intelligent Systems EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 10 130221 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Canny Edge Detector Hough Transform Feature-Based

More information

CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION

CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION In this chapter we will discuss the process of disparity computation. It plays an important role in our caricature system because all 3D coordinates of nodes

More information

Other Linear Filters CS 211A

Other Linear Filters CS 211A Other Linear Filters CS 211A Slides from Cornelia Fermüller and Marc Pollefeys Edge detection Convert a 2D image into a set of curves Extracts salient features of the scene More compact than pixels Origin

More information

Multivariate Calculus Review Problems for Examination Two

Multivariate Calculus Review Problems for Examination Two Multivariate Calculus Review Problems for Examination Two Note: Exam Two is on Thursday, February 28, class time. The coverage is multivariate differential calculus and double integration: sections 13.3,

More information

Announcements. Edges. Last Lecture. Gradients: Numerical Derivatives f(x) Edge Detection, Lines. Intro Computer Vision. CSE 152 Lecture 10

Announcements. Edges. Last Lecture. Gradients: Numerical Derivatives f(x) Edge Detection, Lines. Intro Computer Vision. CSE 152 Lecture 10 Announcements Assignment 2 due Tuesday, May 4. Edge Detection, Lines Midterm: Thursday, May 6. Introduction to Computer Vision CSE 152 Lecture 10 Edges Last Lecture 1. Object boundaries 2. Surface normal

More information

June Wesley, Cambridge Univ. Press,

June Wesley, Cambridge Univ. Press, [6] P Dupuis and J Oliensis. An optimal control formulation and related numerical methods for a problem in shape reconstruction. Ann. Appl. Probab., 4():87{ 346, 994. [7] B K P Horn. Obtaining shape from

More information

CHAPTER 9. Classification Scheme Using Modified Photometric. Stereo and 2D Spectra Comparison

CHAPTER 9. Classification Scheme Using Modified Photometric. Stereo and 2D Spectra Comparison CHAPTER 9 Classification Scheme Using Modified Photometric Stereo and 2D Spectra Comparison 9.1. Introduction In Chapter 8, even we combine more feature spaces and more feature generators, we note that

More information

CS201 Computer Vision Lect 4 - Image Formation

CS201 Computer Vision Lect 4 - Image Formation CS201 Computer Vision Lect 4 - Image Formation John Magee 9 September, 2014 Slides courtesy of Diane H. Theriault Question of the Day: Why is Computer Vision hard? Something to think about from our view

More information

Stereo. Outline. Multiple views 3/29/2017. Thurs Mar 30 Kristen Grauman UT Austin. Multi-view geometry, matching, invariant features, stereo vision

Stereo. Outline. Multiple views 3/29/2017. Thurs Mar 30 Kristen Grauman UT Austin. Multi-view geometry, matching, invariant features, stereo vision Stereo Thurs Mar 30 Kristen Grauman UT Austin Outline Last time: Human stereopsis Epipolar geometry and the epipolar constraint Case example with parallel optical axes General case with calibrated cameras

More information