Modeling Light. Michal Havlik : Computational Photography Alexei Efros, CMU, Fall 2007

Size: px
Start display at page:

Download "Modeling Light. Michal Havlik : Computational Photography Alexei Efros, CMU, Fall 2007"

Transcription

1 Modeling Light Michal Havlik : Computational Photography Alexei Efros, CMU, Fall 2007

2 The Plenoptic Function Figure by Leonard McMillan Q: What is the set of all things that we can ever see? A: The Plenoptic Function (Adelson & Bergen) Let s start with a stationary person and try to parameterize everything that he can see

3 Grayscale snapshot is intensity of light P(θ,φ) Seen from a single view point At a single time Averaged over the wavelengths of the visible spectrum (can also do P(x,y), but spherical coordinate are nicer)

4 Color snapshot is intensity of light P(θ,φ,λ) Seen from a single view point At a single time As a function of wavelength

5 A movie is intensity of light P(θ,φ,λ,t) Seen from a single view point Over time As a function of wavelength

6 Holographic movie is intensity of light Seen from ANY viewpoint Over time As a function of wavelength P(θ,φ,λ,t,V X,V Y,V Z )

7 The Plenoptic Function P(θ,φ,λ,t,V X,V Y,V Z ) Can reconstruct every possible view, at every moment, from every position, at every wavelength Contains every photograph, every movie, everything that anyone has ever seen! it completely captures our visual reality! Not bad for a function

8 Sampling Plenoptic Function (top view) Just lookup -- Quicktime VR

9 Ray Let s not worry about time and color: 5D 3D position 2D direction P(θ,φ, θ,φ,v X,V Y,V Z ) Slide by Rick Szeliski and Michael Cohen

10 How can we use this? Lighting No Change in Radiance Surface Camera

11 Ray Reuse Infinite line Assume light is constant (vacuum) 4D 2D direction 2D position non-dispersive medium Slide by Rick Szeliski and Michael Cohen

12 Only need plenoptic surface

13 Synthesizing novel views Slide by Rick Szeliski and Michael Cohen

14 Lumigraph / Lightfield Outside convex space 4D Empty Stuff Slide by Rick Szeliski and Michael Cohen

15 Lumigraph - Organization 2D position 2D direction θ s Slide by Rick Szeliski and Michael Cohen

16 Lumigraph - Organization 2D position 2D position s u 2 plane parameterization Slide by Rick Szeliski and Michael Cohen

17 Lumigraph - Organization 2D position 2D position t s,t s,t u,v v 2 plane parameterization s u u,v Slide by Rick Szeliski and Michael Cohen

18 Lumigraph - Organization Hold s,t constant Let u,v vary An image s,t u,v Slide by Rick Szeliski and Michael Cohen

19 Lumigraph / Lightfield

20 Lumigraph - Capture Idea 1 Move camera carefully over s,t plane Gantry see Lightfield paper s,t u,v Slide by Rick Szeliski and Michael Cohen

21 Lumigraph - Capture Idea 2 Move camera anywhere Rebinning see Lumigraph paper s,t u,v Slide by Rick Szeliski and Michael Cohen

22 Lumigraph - Rendering For each output pixel determine s,t,u,v either use closest discrete RGB interpolate near values s u Slide by Rick Szeliski and Michael Cohen

23 Lumigraph - Rendering Nearest closest s closest u draw it Blend 16 nearest quadrilinear interpolation s u Slide by Rick Szeliski and Michael Cohen

24 Plenoptic Function/Light Field/Lumigraph

25 Omnidirectional Panoramas

26 Omnidirectional Panoramas

27 Omnidirectional Panoramas

28 Omnidirectional Panoramas

29 Omnidirectional Panoramas

30 Omnidirectional Panoramas

31 Stanford multi-camera array pixels 30 fps 128 cameras synchronized timing continuous streaming flexible arrangement

32 Ways to use large camera arrays widely spaced light field capture tightly packed high-performance imaging intermediate spacing synthetic aperture photography

33 Synthetic Aperture Image

34 Example using 45 cameras [Vaish CVPR 2004]

35

36 Tiled camera array Can we match the image quality of a cinema camera? world s largest video camera no parallax for distant objects poor lenses limit image quality seamless mosaicing isn t hard

37 Tiled panoramic image (before geometric or color calibration)

38 Tiled panoramic image (after calibration and blending)

39 Tiled camera array Can we match the image quality of a cinema camera? world s largest video camera no parallax for distant objects poor lenses limit image quality seamless mosaicing isn t hard per-camera exposure metering HDR within and between tiles

40 same exposure in all cameras individually metered checkerboard of exposures

41 High-performance photography as multidimensional sampling spatial resolution field of view frame rate dynamic range bits of precision depth of field focus setting color sensitivity

42 Spacetime aperture shaping shorten exposure time to freeze motion dark stretch contrast to restore level noisy increase (synthetic) aperture to capture more light decreases depth of field

43 center of aperture: few cameras, long exposure high depth of field, low noise, but action is blurred periphery of aperture: many cameras, short exposure freezes action, low noise, but low depth of field

44

45

46 Light field photography using a handheld plenoptic camera Ren Ng, Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz and Pat Hanrahan

47 Conventional versus light field camera

48 Conventional versus light field camera uv-plane st-plane

49 Prototype camera Contax medium format camera Kodak 16-megapixel sensor Adaptive Optics microlens array 125µ square-sided microlenses pixels lenses = pixels per lens

50

51 Digitally stopping-down Σ Σ stopping down = summing only the central portion of each microlens

52 Digital refocusing Σ Σ refocusing = summing windows extracted from several microlenses

53 Example of digital refocusing

54 Digitally moving the observer Σ moving the observer = moving the window we extract from the microlenses Σ

55 Example of moving the observer

56 Moving backward and forward

Modeling Light. Michal Havlik : Computational Photography Alexei Efros, CMU, Fall 2011

Modeling Light. Michal Havlik : Computational Photography Alexei Efros, CMU, Fall 2011 Modeling Light Michal Havlik 15-463: Computational Photography Alexei Efros, CMU, Fall 2011 What is light? Electromagnetic radiation (EMR) moving along rays in space R(λ) is EMR, measured in units of power

More information

Modeling Light. Michal Havlik

Modeling Light. Michal Havlik Modeling Light Michal Havlik 15-463: Computational Photography Alexei Efros, CMU, Spring 2010 What is light? Electromagnetic radiation (EMR) moving along rays in space R( ) is EMR, measured in units of

More information

Modeling Light. Slides from Alexei A. Efros and others

Modeling Light. Slides from Alexei A. Efros and others Project 3 Results http://www.cs.brown.edu/courses/cs129/results/proj3/jcmace/ http://www.cs.brown.edu/courses/cs129/results/proj3/damoreno/ http://www.cs.brown.edu/courses/cs129/results/proj3/taox/ Stereo

More information

Modeling Light. Michal Havlik

Modeling Light. Michal Havlik Modeling Light Michal Havlik 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 What is light? Electromagnetic radiation (EMR) moving along rays in space R(λ) is EMR, measured in units of power

More information

Modeling Light. On Simulating the Visual Experience

Modeling Light. On Simulating the Visual Experience Modeling Light 15-463: Rendering and Image Processing Alexei Efros On Simulating the Visual Experience Just feed the eyes the right data No one will know the difference! Philosophy: Ancient question: Does

More information

Light Field Spring

Light Field Spring Light Field 2015 Spring Recall: Light is Electromagnetic radiation (EMR) moving along rays in space R(l) is EMR, measured in units of power (watts) l is wavelength Useful things: Light travels in straight

More information

CSCI 1290: Comp Photo

CSCI 1290: Comp Photo CSCI 1290: Comp Photo Fall 2018 @ Brown University James Tompkin Many slides thanks to James Hays old CS 129 course, along with all of its acknowledgements. What do we see? 3D world 2D image Point of observation

More information

But, vision technology falls short. and so does graphics. Image Based Rendering. Ray. Constant radiance. time is fixed. 3D position 2D direction

But, vision technology falls short. and so does graphics. Image Based Rendering. Ray. Constant radiance. time is fixed. 3D position 2D direction Computer Graphics -based rendering Output Michael F. Cohen Microsoft Research Synthetic Camera Model Computer Vision Combined Output Output Model Real Scene Synthetic Camera Model Real Cameras Real Scene

More information

Image or Object? Is this real?

Image or Object? Is this real? Image or Object? Michael F. Cohen Microsoft Is this real? Photo by Patrick Jennings (patrick@synaptic.bc.ca), Copyright 1995, 96, 97 Whistler B. C. Canada Modeling, Rendering, and Lighting 1 A mental model?

More information

Computational Photography

Computational Photography Computational Photography Matthias Zwicker University of Bern Fall 2010 Today Light fields Introduction Light fields Signal processing analysis Light field cameras Application Introduction Pinhole camera

More information

Image-Based Rendering

Image-Based Rendering Image-Based Rendering COS 526, Fall 2016 Thomas Funkhouser Acknowledgments: Dan Aliaga, Marc Levoy, Szymon Rusinkiewicz What is Image-Based Rendering? Definition 1: the use of photographic imagery to overcome

More information

More and More on Light Fields. Last Lecture

More and More on Light Fields. Last Lecture More and More on Light Fields Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 4 Last Lecture Re-review with emphasis on radiometry Mosaics & Quicktime VR The Plenoptic function The main

More information

Plenoptic camera and its Applications

Plenoptic camera and its Applications Aum Sri Sairam Plenoptic camera and its Applications Agenda: 1. Introduction 2. Single lens stereo design 3. Plenoptic camera design 4. Depth estimation 5. Synthetic refocusing 6. Fourier slicing 7. References

More information

Coding and Modulation in Cameras

Coding and Modulation in Cameras Mitsubishi Electric Research Laboratories Raskar 2007 Coding and Modulation in Cameras Ramesh Raskar with Ashok Veeraraghavan, Amit Agrawal, Jack Tumblin, Ankit Mohan Mitsubishi Electric Research Labs

More information

Image-Based Modeling and Rendering

Image-Based Modeling and Rendering Image-Based Modeling and Rendering Richard Szeliski Microsoft Research IPAM Graduate Summer School: Computer Vision July 26, 2013 How far have we come? Light Fields / Lumigraph - 1996 Richard Szeliski

More information

Computational Photography: Real Time Plenoptic Rendering

Computational Photography: Real Time Plenoptic Rendering Computational Photography: Real Time Plenoptic Rendering Andrew Lumsdaine, Georgi Chunev Indiana University Todor Georgiev Adobe Systems Who was at the Keynote Yesterday? 2 Overview Plenoptic cameras Rendering

More information

Jingyi Yu CISC 849. Department of Computer and Information Science

Jingyi Yu CISC 849. Department of Computer and Information Science Digital Photography and Videos Jingyi Yu CISC 849 Light Fields, Lumigraph, and Image-based Rendering Pinhole Camera A camera captures a set of rays A pinhole camera captures a set of rays passing through

More information

Image-based modeling (IBM) and image-based rendering (IBR)

Image-based modeling (IBM) and image-based rendering (IBR) Image-based modeling (IBM) and image-based rendering (IBR) CS 248 - Introduction to Computer Graphics Autumn quarter, 2005 Slides for December 8 lecture The graphics pipeline modeling animation rendering

More information

The Focused Plenoptic Camera. Lightfield photographers, focus your cameras! Karl Marx

The Focused Plenoptic Camera. Lightfield photographers, focus your cameras! Karl Marx The Focused Plenoptic Camera Lightfield photographers, focus your cameras! Karl Marx Plenoptic Camera, Adelson 1992 Main lens focused on microlenses Plenoptic Camera, Adelson 1992 Microlenses focused on

More information

A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India

A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India Keshav Mahavidyalaya, University of Delhi, Delhi, India Abstract

More information

Focal stacks and lightfields

Focal stacks and lightfields Focal stacks and lightfields http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 11 Course announcements Homework 3 is out. - Due October 12 th.

More information

Real-time Generation and Presentation of View-dependent Binocular Stereo Images Using a Sequence of Omnidirectional Images

Real-time Generation and Presentation of View-dependent Binocular Stereo Images Using a Sequence of Omnidirectional Images Real-time Generation and Presentation of View-dependent Binocular Stereo Images Using a Sequence of Omnidirectional Images Abstract This paper presents a new method to generate and present arbitrarily

More information

Computational Photography

Computational Photography Computational Photography Photography and Imaging Michael S. Brown Brown - 1 Part 1 Overview Photography Preliminaries Traditional Film Imaging (Camera) Part 2 General Imaging 5D Plenoptic Function (McMillan)

More information

Image Based Rendering. D.A. Forsyth, with slides from John Hart

Image Based Rendering. D.A. Forsyth, with slides from John Hart Image Based Rendering D.A. Forsyth, with slides from John Hart Topics Mosaics translating cameras reveal extra information, break occlusion Optical flow for very small movements of the camera Explicit

More information

AN O(N 2 LOG(N)) PER PLANE FAST DISCRETE FOCAL STACK TRANSFORM

AN O(N 2 LOG(N)) PER PLANE FAST DISCRETE FOCAL STACK TRANSFORM AN O(N 2 LOG(N)) PER PLANE FAST DISCRETE FOCAL STACK TRANSFORM Fernando Pérez Nava +, Jonás Philipp Lüke + Departamento de Estadística, Investigación Operativa y Computación Departamento de Física Fundamental

More information

Announcements. Light. Properties of light. Light. Project status reports on Wednesday. Readings. Today. Readings Szeliski, 2.2, 2.3.

Announcements. Light. Properties of light. Light. Project status reports on Wednesday. Readings. Today. Readings Szeliski, 2.2, 2.3. Announcements Project status reports on Wednesday prepare 5 minute ppt presentation should contain: problem statement (1 slide) description of approach (1 slide) some images (1 slide) current status +

More information

Re-live the Movie Matrix : From Harry Nyquist to Image-Based Rendering. Tsuhan Chen Carnegie Mellon University Pittsburgh, USA

Re-live the Movie Matrix : From Harry Nyquist to Image-Based Rendering. Tsuhan Chen Carnegie Mellon University Pittsburgh, USA Re-live the Movie Matrix : From Harry Nyquist to Image-Based Rendering Tsuhan Chen tsuhan@cmu.edu Carnegie Mellon University Pittsburgh, USA Some History IEEE Multimedia Signal Processing (MMSP) Technical

More information

The 7d plenoptic function, indexing all light.

The 7d plenoptic function, indexing all light. Previous Lecture The 7d plenoptic function, indexing all light. Lightfields: a 4d (not 5d!) data structure which captures all outgoing light from a region and permits reconstruction of arbitrary synthetic

More information

DEPTH, STEREO AND FOCUS WITH LIGHT-FIELD CAMERAS

DEPTH, STEREO AND FOCUS WITH LIGHT-FIELD CAMERAS DEPTH, STEREO AND FOCUS WITH LIGHT-FIELD CAMERAS CINEC 2014 Frederik Zilly, Head of Group Computational Imaging & Algorithms Moving Picture Technologies Department, Fraunhofer IIS Fraunhofer, Frederik

More information

Radiance Photography. Todor Georgiev Adobe Systems. Andrew Lumsdaine Indiana University

Radiance Photography. Todor Georgiev Adobe Systems. Andrew Lumsdaine Indiana University Radiance Photography Todor Georgiev Adobe Systems Andrew Lumsdaine Indiana University Course Goals Overview of radiance (aka lightfield) photography Mathematical treatment of theory and computation Hands

More information

Image-Based Modeling and Rendering

Image-Based Modeling and Rendering Traditional Computer Graphics Image-Based Modeling and Rendering Thomas Funkhouser Princeton University COS 426 Guest Lecture Spring 2003 How would you model and render this scene? (Jensen) How about this

More information

Structure from Motion and Multi- view Geometry. Last lecture

Structure from Motion and Multi- view Geometry. Last lecture Structure from Motion and Multi- view Geometry Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 5 Last lecture S. J. Gortler, R. Grzeszczuk, R. Szeliski,M. F. Cohen The Lumigraph, SIGGRAPH,

More information

Computational Cameras: Exploiting Spatial- Angular Temporal Tradeoffs in Photography

Computational Cameras: Exploiting Spatial- Angular Temporal Tradeoffs in Photography Mitsubishi Electric Research Labs (MERL) Computational Cameras Computational Cameras: Exploiting Spatial- Angular Temporal Tradeoffs in Photography Amit Agrawal Mitsubishi Electric Research Labs (MERL)

More information

VIDEO FOR VIRTUAL REALITY LIGHT FIELD BASICS JAMES TOMPKIN

VIDEO FOR VIRTUAL REALITY LIGHT FIELD BASICS JAMES TOMPKIN VIDEO FOR VIRTUAL REALITY LIGHT FIELD BASICS JAMES TOMPKIN WHAT IS A LIGHT FIELD? Light field seems to have turned into a catch-all term for many advanced camera/display technologies. WHAT IS A LIGHT FIELD?

More information

Image-Based Modeling and Rendering. Image-Based Modeling and Rendering. Final projects IBMR. What we have learnt so far. What IBMR is about

Image-Based Modeling and Rendering. Image-Based Modeling and Rendering. Final projects IBMR. What we have learnt so far. What IBMR is about Image-Based Modeling and Rendering Image-Based Modeling and Rendering MIT EECS 6.837 Frédo Durand and Seth Teller 1 Some slides courtesy of Leonard McMillan, Wojciech Matusik, Byong Mok Oh, Max Chen 2

More information

Light Fields. Johns Hopkins Department of Computer Science Course : Rendering Techniques, Professor: Jonathan Cohen

Light Fields. Johns Hopkins Department of Computer Science Course : Rendering Techniques, Professor: Jonathan Cohen Light Fields Light Fields By Levoy and Hanrahan, SIGGRAPH 96 Representation for sampled plenoptic function stores data about visible light at various positions and directions Created from set of images

More information

http://www.diva-portal.org This is the published version of a paper presented at 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), Stockholm Helsinki Stockholm,

More information

Volumetric Scene Reconstruction from Multiple Views

Volumetric Scene Reconstruction from Multiple Views Volumetric Scene Reconstruction from Multiple Views Chuck Dyer University of Wisconsin dyer@cs cs.wisc.edu www.cs cs.wisc.edu/~dyer Image-Based Scene Reconstruction Goal Automatic construction of photo-realistic

More information

DEPTH AND ANGULAR RESOLUTION IN PLENOPTIC CAMERAS. M. Damghanian, R. Olsson, M. Sjöström

DEPTH AND ANGULAR RESOLUTION IN PLENOPTIC CAMERAS. M. Damghanian, R. Olsson, M. Sjöström This material is published in the open archive of Mid Sweden University DIVA http://miun.diva-portal.org to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein

More information

Image-Based Lighting. Eirik Holmøyvik. with a lot of slides donated by Paul Debevec

Image-Based Lighting. Eirik Holmøyvik. with a lot of slides donated by Paul Debevec Image-Based Lighting Eirik Holmøyvik with a lot of slides donated by Paul Debevec 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Inserting Synthetic Objects Why does this look so bad? Wrong

More information

Multiple View Geometry

Multiple View Geometry Multiple View Geometry Martin Quinn with a lot of slides stolen from Steve Seitz and Jianbo Shi 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Our Goal The Plenoptic Function P(θ,φ,λ,t,V

More information

Computational Methods for Radiance. Render the full variety offered by the direct observation of objects. (Computationally).

Computational Methods for Radiance. Render the full variety offered by the direct observation of objects. (Computationally). Computational Methods for Radiance Render the full variety offered by the direct observation of objects. (Computationally). Methods for Plenoptic 1.0 Computing with Radiance Goal: Render the full variety

More information

Image-Based Lighting : Computational Photography Alexei Efros, CMU, Fall Eirik Holmøyvik. with a lot of slides donated by Paul Debevec

Image-Based Lighting : Computational Photography Alexei Efros, CMU, Fall Eirik Holmøyvik. with a lot of slides donated by Paul Debevec Image-Based Lighting Eirik Holmøyvik with a lot of slides donated by Paul Debevec 15-463: Computational Photography Alexei Efros, CMU, Fall 2011 Inserting Synthetic Objects Why does this look so bad? Wrong

More information

Dense Lightfield Disparity Estimation using Total Variation Regularization

Dense Lightfield Disparity Estimation using Total Variation Regularization Dense Lightfield Disparity Estimation using Total Variation Regularization Nuno Barroso Monteiro 1,2, João Pedro Barreto 2, and José Gaspar 1 1 Institute for Systems and Robotics, Univ. of Lisbon, Portugal

More information

3D Particle Position Reconstruction Accuracy in Plenoptic PIV

3D Particle Position Reconstruction Accuracy in Plenoptic PIV AIAA SciTech 13-17 January 2014, National Harbor, Maryland 52nd Aerospace Sciences Meeting AIAA 2014-0398 3D Particle Position Reconstruction Accuracy in Plenoptic PIV Timothy W. Fahringer and Brian S.

More information

An Approach on Hardware Design for Computationally Intensive Image Processing Applications based on Light Field Refocusing Algorithm

An Approach on Hardware Design for Computationally Intensive Image Processing Applications based on Light Field Refocusing Algorithm An Approach on Hardware Design for Computationally Intensive Image Processing Applications based on Light Field Refocusing Algorithm Jiayuan Meng, Dee A.B. Weikle, Greg Humphreys, Kevin Skadron Dept. of

More information

Homographies and RANSAC

Homographies and RANSAC Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position

More information

Title: The Future of Photography is Computational Photography. Subtitle: 100 years after invention, Integral Photography becomes feasible

Title: The Future of Photography is Computational Photography. Subtitle: 100 years after invention, Integral Photography becomes feasible Title: The Future of Photography is Computational Photography Subtitle: 100 years after invention, Integral Photography becomes feasible Adobe customers are creative and demanding. They expect to use our

More information

Feature Matching and RANSAC

Feature Matching and RANSAC Feature Matching and RANSAC Recognising Panoramas. [M. Brown and D. Lowe,ICCV 2003] [Brown, Szeliski, Winder, CVPR 2005] with a lot of slides stolen from Steve Seitz, Rick Szeliski, A. Efros Introduction

More information

New Frontier in Visual Communication and Networking: Multi-View Imaging. Professor Tsuhan Chen 陳祖翰

New Frontier in Visual Communication and Networking: Multi-View Imaging. Professor Tsuhan Chen 陳祖翰 New Frontier in Visual Communication and Networking: Multi-View Imaging Professor Tsuhan Chen 陳祖翰 tsuhan@cmu.edu Carnegie Mellon University Pittsburgh, USA A 10-Year Journey IEEE Multimedia Signal Processing

More information

Image-Based Lighting

Image-Based Lighting Image-Based Lighting Eirik Holmøyvik CS194: Image Manipulation & Computational Photography with a lot of slides Alexei Efros, UC Berkeley, Fall 2014 donated by Paul Debevec Inserting Synthetic Objects

More information

Image Based Rendering

Image Based Rendering Image Based Rendering an overview Photographs We have tools that acquire and tools that display photographs at a convincing quality level 2 1 3 4 2 5 6 3 7 8 4 9 10 5 Photographs We have tools that acquire

More information

The Effect of Grid Resolution on the Accuracy of Tomographic Reconstruction Using a Plenoptic Camera

The Effect of Grid Resolution on the Accuracy of Tomographic Reconstruction Using a Plenoptic Camera 51st AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition 7-1 January 13, Grapevine (Dallas/Ft. Worth Region), Texas AIAA 13-39 The Effect of Grid Resolution on the

More information

Automatic Image Alignment (feature-based)

Automatic Image Alignment (feature-based) Automatic Image Alignment (feature-based) Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Today s lecture Feature

More information

Morphable 3D-Mosaics: a Hybrid Framework for Photorealistic Walkthroughs of Large Natural Environments

Morphable 3D-Mosaics: a Hybrid Framework for Photorealistic Walkthroughs of Large Natural Environments Morphable 3D-Mosaics: a Hybrid Framework for Photorealistic Walkthroughs of Large Natural Environments Nikos Komodakis and Georgios Tziritas Computer Science Department, University of Crete E-mails: {komod,

More information

CSc Topics in Computer Graphics 3D Photography

CSc Topics in Computer Graphics 3D Photography CSc 83010 Topics in Computer Graphics 3D Photography Tuesdays 11:45-1:45 1:45 Room 3305 Ioannis Stamos istamos@hunter.cuny.edu Office: 1090F, Hunter North (Entrance at 69 th bw/ / Park and Lexington Avenues)

More information

Capturing and Rendering With Incident Light Fields

Capturing and Rendering With Incident Light Fields EUROGRAPHICS 0x / Per Christensen and Daniel Cohen-Or (Guest Editors) Volume 0 (1981), Number 0 Capturing and Rendering With Incident Light Fields J. Unger, 1 A. Wenger, 2 T. Hawkins, 2 A. Gardner 2 and

More information

Real Time Rendering. CS 563 Advanced Topics in Computer Graphics. Songxiang Gu Jan, 31, 2005

Real Time Rendering. CS 563 Advanced Topics in Computer Graphics. Songxiang Gu Jan, 31, 2005 Real Time Rendering CS 563 Advanced Topics in Computer Graphics Songxiang Gu Jan, 31, 2005 Introduction Polygon based rendering Phong modeling Texture mapping Opengl, Directx Point based rendering VTK

More information

Depth Estimation with a Plenoptic Camera

Depth Estimation with a Plenoptic Camera Depth Estimation with a Plenoptic Camera Steven P. Carpenter 1 Auburn University, Auburn, AL, 36849 The plenoptic camera is a tool capable of recording significantly more data concerning a particular image

More information

Hybrid Rendering for Collaborative, Immersive Virtual Environments

Hybrid Rendering for Collaborative, Immersive Virtual Environments Hybrid Rendering for Collaborative, Immersive Virtual Environments Stephan Würmlin wuermlin@inf.ethz.ch Outline! Rendering techniques GBR, IBR and HR! From images to models! Novel view generation! Putting

More information

Lecture 15: Image-Based Rendering and the Light Field. Kayvon Fatahalian CMU : Graphics and Imaging Architectures (Fall 2011)

Lecture 15: Image-Based Rendering and the Light Field. Kayvon Fatahalian CMU : Graphics and Imaging Architectures (Fall 2011) Lecture 15: Image-Based Rendering and the Light Field Kayvon Fatahalian CMU 15-869: Graphics and Imaging Architectures (Fall 2011) Demo (movie) Royal Palace: Madrid, Spain Image-based rendering (IBR) So

More information

A Warping-based Refinement of Lumigraphs

A Warping-based Refinement of Lumigraphs A Warping-based Refinement of Lumigraphs Wolfgang Heidrich, Hartmut Schirmacher, Hendrik Kück, Hans-Peter Seidel Computer Graphics Group University of Erlangen heidrich,schirmacher,hkkueck,seidel@immd9.informatik.uni-erlangen.de

More information

A million pixels, a million polygons. Which is heavier? François X. Sillion. imagis* Grenoble, France

A million pixels, a million polygons. Which is heavier? François X. Sillion. imagis* Grenoble, France A million pixels, a million polygons. Which is heavier? François X. Sillion imagis* Grenoble, France *A joint research project of CNRS, INRIA, INPG and UJF MAGIS Why this question? Evolution of processing

More information

More Mosaic Madness. CS194: Image Manipulation & Computational Photography. Steve Seitz and Rick Szeliski. Jeffrey Martin (jeffrey-martin.

More Mosaic Madness. CS194: Image Manipulation & Computational Photography. Steve Seitz and Rick Szeliski. Jeffrey Martin (jeffrey-martin. More Mosaic Madness Jeffrey Martin (jeffrey-martin.com) CS194: Image Manipulation & Computational Photography with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 2018 Steve Seitz and Rick

More information

Globally Consistent Depth Labeling of 4D Light Fields

Globally Consistent Depth Labeling of 4D Light Fields Globally Consistent Depth Labeling of 4D Light Fields Sven Wanner and Bastian Goldluecke Heidelberg Collaboratory for Image Processing Abstract We present a novel paradigm to deal with depth reconstruction

More information

DD2423 Image Analysis and Computer Vision IMAGE FORMATION. Computational Vision and Active Perception School of Computer Science and Communication

DD2423 Image Analysis and Computer Vision IMAGE FORMATION. Computational Vision and Active Perception School of Computer Science and Communication DD2423 Image Analysis and Computer Vision IMAGE FORMATION Mårten Björkman Computational Vision and Active Perception School of Computer Science and Communication November 8, 2013 1 Image formation Goal:

More information

The Plenoptic videos: Capturing, Rendering and Compression. Chan, SC; Ng, KT; Gan, ZF; Chan, KL; Shum, HY.

The Plenoptic videos: Capturing, Rendering and Compression. Chan, SC; Ng, KT; Gan, ZF; Chan, KL; Shum, HY. Title The Plenoptic videos: Capturing, Rendering and Compression Author(s) Chan, SC; Ng, KT; Gan, ZF; Chan, KL; Shum, HY Citation IEEE International Symposium on Circuits and Systems Proceedings, Vancouver,

More information

Light. Properties of light. What is light? Today What is light? How do we measure it? How does light propagate? How does light interact with matter?

Light. Properties of light. What is light? Today What is light? How do we measure it? How does light propagate? How does light interact with matter? Light Properties of light Today What is light? How do we measure it? How does light propagate? How does light interact with matter? by Ted Adelson Readings Andrew Glassner, Principles of Digital Image

More information

Capturing and View-Dependent Rendering of Billboard Models

Capturing and View-Dependent Rendering of Billboard Models Capturing and View-Dependent Rendering of Billboard Models Oliver Le, Anusheel Bhushan, Pablo Diaz-Gutierrez and M. Gopi Computer Graphics Lab University of California, Irvine Abstract. In this paper,

More information

CS5670: Computer Vision

CS5670: Computer Vision CS5670: Computer Vision Noah Snavely Light & Perception Announcements Quiz on Tuesday Project 3 code due Monday, April 17, by 11:59pm artifact due Wednesday, April 19, by 11:59pm Can we determine shape

More information

Comparing Volumetric Reconstruction Algorithms for Plenoptic-PIV

Comparing Volumetric Reconstruction Algorithms for Plenoptic-PIV Comparing Volumetric Reconstruction Algorithms for Plenoptic-PIV Timothy W. Fahringer and Brian S. Thurow Auburn University, AL, 36849, USA A new algorithm for reconstruction of 3D particle fields from

More information

Image-Based Lighting. Inserting Synthetic Objects

Image-Based Lighting. Inserting Synthetic Objects Image-Based Lighting 15-463: Rendering and Image Processing Alexei Efros with a lot of slides donated by Paul Debevec Inserting Synthetic Objects Why does this look so bad? Wrong camera orientation Wrong

More information

Tomographic Reconstruction of a 3-D Flow Field Using a Plenoptic Camera

Tomographic Reconstruction of a 3-D Flow Field Using a Plenoptic Camera 42nd AIAA Fluid Dynamics Conference and Exhibit 25-28 June 2012, New Orleans, Louisiana AIAA 2012-2826 Tomographic Reconstruction of a 3-D Flow Field Using a Plenoptic Camera Timothy W. Fahringer and Brian

More information

A unified approach for motion analysis and view synthesis Λ

A unified approach for motion analysis and view synthesis Λ A unified approach for motion analysis and view synthesis Λ Alex Rav-Acha Shmuel Peleg School of Computer Science and Engineering The Hebrew University of Jerusalem 994 Jerusalem, Israel Email: falexis,pelegg@cs.huji.ac.il

More information

Automatic Image Alignment

Automatic Image Alignment Automatic Image Alignment Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 Live Homography DEMO Check out panoramio.com

More information

Consider a partially transparent object that is illuminated with two lights, one visible from each side of the object. Start with a ray from the eye

Consider a partially transparent object that is illuminated with two lights, one visible from each side of the object. Start with a ray from the eye Ray Tracing What was the rendering equation? Motivate & list the terms. Relate the rendering equation to forward ray tracing. Why is forward ray tracing not good for image formation? What is the difference

More information

Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi

Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Image Stitching Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish

More information

IMAGE-BASED RENDERING TECHNIQUES FOR APPLICATION IN VIRTUAL ENVIRONMENTS

IMAGE-BASED RENDERING TECHNIQUES FOR APPLICATION IN VIRTUAL ENVIRONMENTS IMAGE-BASED RENDERING TECHNIQUES FOR APPLICATION IN VIRTUAL ENVIRONMENTS Xiaoyong Sun A Thesis submitted to the Faculty of Graduate and Postdoctoral Studies in partial fulfillment of the requirements for

More information

Image Warping and Mosacing

Image Warping and Mosacing Image Warping and Mosacing 15-463: Rendering and Image Processing Alexei Efros with a lot of slides stolen from Steve Seitz and Rick Szeliski Today Mosacs Image Warping Homographies Programming Assignment

More information

Plenoptic Cameras. Bastian Goldlücke, Oliver Klehm, Sven Wanner, and Elmar Eisemann. 5.1 Introduction

Plenoptic Cameras. Bastian Goldlücke, Oliver Klehm, Sven Wanner, and Elmar Eisemann. 5.1 Introduction Plenoptic Cameras Bastian Goldlücke, Oliver Klehm, Sven Wanner, and Elmar Eisemann 5.1 Introduction The light field, as defined by Gershun in 1936 [Gershun 36] describes the radiance traveling in every

More information

Image-Based Rendering and Modeling. IBR Approaches for View Synthesis

Image-Based Rendering and Modeling. IBR Approaches for View Synthesis Image-Based Rendering and Modeling l Image-based rendering (IBR): A scene is represented as a collection of images l 3D model-based rendering (MBR): A scene is represented by a 3D model plus texture maps

More information

PAPER Three-Dimensional Scene Walkthrough System Using Multiple Acentric Panorama View (APV) Technique

PAPER Three-Dimensional Scene Walkthrough System Using Multiple Acentric Panorama View (APV) Technique IEICE TRANS. INF. & SYST., VOL.E86 D, NO.1 JANUARY 2003 117 PAPER Three-Dimensional Scene Walkthrough System Using Multiple Acentric Panorama View (APV) Technique Ping-Hsien LIN and Tong-Yee LEE, Nonmembers

More information

Image-Based Rendering. Johns Hopkins Department of Computer Science Course : Rendering Techniques, Professor: Jonathan Cohen

Image-Based Rendering. Johns Hopkins Department of Computer Science Course : Rendering Techniques, Professor: Jonathan Cohen Image-Based Rendering Image-Based Rendering What is it? Still a difficult question to answer Uses images (photometric( info) as key component of model representation What s Good about IBR Model acquisition

More information

Full-Resolution Depth Map Estimation from an Aliased Plenoptic Light Field

Full-Resolution Depth Map Estimation from an Aliased Plenoptic Light Field Full-Resolution Depth Map Estimation from an Aliased Plenoptic Light Field Tom E. Bishop and Paolo Favaro Department of Engineering and Physical Sciences Heriot-Watt University, Edinburgh, UK {t.e.bishop,p.favaro}@hw.ac.uk

More information

Optimizing an Inverse Warper

Optimizing an Inverse Warper Optimizing an Inverse Warper by Robert W. Marcato, Jr. Submitted to the Department of Electrical Engineering and Computer Science in Partial Fulfillment of the Requirements for the Degrees of Bachelor

More information

Mosaics, Plenoptic Function, and Light Field Rendering. Last Lecture

Mosaics, Plenoptic Function, and Light Field Rendering. Last Lecture Mosaics, Plenoptic Function, and Light Field Rendering Topics in Image-ased Modeling and Rendering CSE291 J00 Lecture 3 Last Lecture Camera Models Pinhole perspective Affine/Orthographic models Homogeneous

More information

Acquiring 4D Light Fields of Self-Luminous Light Sources Using Programmable Filter

Acquiring 4D Light Fields of Self-Luminous Light Sources Using Programmable Filter Acquiring 4D Light Fields of Self-Luminous Light Sources Using Programmable Filter Motohiro Nakamura 1, Takahiro Okabe 1, and Hendrik P. A. Lensch 2 1 Kyushu Institute of Technology 2 Tübingen University

More information

Automatic Image Alignment

Automatic Image Alignment Automatic Image Alignment with a lot of slides stolen from Steve Seitz and Rick Szeliski Mike Nese CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2018 Live Homography

More information

Using Planar Point Correspondence to Calibrate Camera Arrays for Light Field Acquisition

Using Planar Point Correspondence to Calibrate Camera Arrays for Light Field Acquisition Using Planar Point Correspondence to Calibrate Camera Arrays for Light Field Acquisition Ashley W. Stewart, Donald G. Dansereau Queensland University of Technology, Australia; Stanford University a.stewart.au@gmail.com,

More information

CS 684 Fall 2005 Image-based Modeling and Rendering. Ruigang Yang

CS 684 Fall 2005 Image-based Modeling and Rendering. Ruigang Yang CS 684 Fall 2005 Image-based Modeling and Rendering Ruigang Yang Administrivia Classes: Monday and Wednesday, 4:00-5:15 PM Instructor: Ruigang Yang ryang@cs.uky.edu Office Hour: Robotics 514D, MW 1500-1600

More information

What have we leaned so far?

What have we leaned so far? What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging What have we learned so far? Image Filtering Image Warping Camera Projection Model Project 2: Panoramic

More information

Scene surface light field representation for realistic lighting

Scene surface light field representation for realistic lighting Scene surface light field representation for realistic lighting Tao Yu Zhong Zhou Jian Hu QinPing Zhao Optical Engineering 52(12), 123103 (December 2013) Scene surface light field representation for realistic

More information

EE795: Computer Vision and Intelligent Systems

EE795: Computer Vision and Intelligent Systems EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry

More information

Linearizing the Plenoptic Space

Linearizing the Plenoptic Space Linearizing the Plenoptic Space Grégoire Nieto1, Frédéric Devernay1, James Crowley2 LJK, Université Grenoble Alpes, France 2 LIG, Université Grenoble Alpes, France 1 1 Goal: synthesize a new view Capture/sample

More information

High Dynamic Range Images

High Dynamic Range Images High Dynamic Range Images Alyosha Efros CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2018 with a lot of slides stolen from Paul Debevec Why HDR? Problem: Dynamic

More information

Image warping and stitching

Image warping and stitching Image warping and stitching Thurs Oct 15 Last time Feature-based alignment 2D transformations Affine fit RANSAC 1 Robust feature-based alignment Extract features Compute putative matches Loop: Hypothesize

More information

Depth Range Accuracy for Plenoptic Cameras

Depth Range Accuracy for Plenoptic Cameras Depth Range Accuracy for Plenoptic Cameras Nuno Barroso Monteiro Institute for Systems and Robotics, University of Lisbon, Portugal Institute for Systems and Robotics, University of Coimbra, Portugal Simão

More information

Video Texture. A.A. Efros

Video Texture. A.A. Efros Video Texture A.A. Efros 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Weather Forecasting for Dummies Let s predict weather: Given today s weather only, we want to know tomorrow s Suppose

More information

Computational Photography

Computational Photography End of Semester is the last lecture of new material Quiz on Friday 4/30 Sample problems are posted on website Computational Photography Final Project Presentations Wednesday May 12 1-5pm, CII 4040 Attendance

More information

Focused Plenoptic Camera and Rendering. Focused Plenoptic Camera and Rendering. Todor Georgiev 1, a) 2, b) and Andrew Lumsdaine

Focused Plenoptic Camera and Rendering. Focused Plenoptic Camera and Rendering. Todor Georgiev 1, a) 2, b) and Andrew Lumsdaine Focused Plenoptic Camera and Rendering Todor Georgiev 1, a) 2, b) and Andrew Lumsdaine 1) Adobe Systems Inc. 2) Indiana University (Dated: 3 March 2010) Plenoptic cameras, constructed with internal microlens

More information