Image-Based Modeling and Rendering
|
|
|
- Marsha Kelley
- 6 years ago
- Views:
Transcription
1 Image-Based Modeling and Rendering Richard Szeliski Microsoft Research IPAM Graduate Summer School: Computer Vision July 26, 2013
2 How far have we come?
3 Light Fields / Lumigraph Richard Szeliski Image-Based Rendering and Modeling 3
4 Environment matting Richard Szeliski Image-Based Rendering and Modeling 4
5 Photo Tourism Richard Szeliski Image-Based Rendering and Modeling 5
6 Ambient Point Clouds Richard Szeliski Image-Based Rendering and Modeling 6
7 Photo Tours Richard Szeliski Image-Based Rendering and Modeling 7 [Kushal et al., 3DIMPVT 2012]
8 The Visual Turing Test Video Richard Szeliski Image-Based Rendering and Modeling 8 [Shan et al., 3DV 2013]
9 Where are we going?
10 A personal retrospective Panoramas and 360 Video Walkthroughs Light Fields and Lumigraphs LDIs, Sprites, and Layered Video Image-Based Modeling Environment Mattes and Matting Photo Tourism and Photosynth Point-Based Rendering and NPR Reflections and Transparency [2012] Richard Szeliski Image-Based Rendering and Modeling 10
11 Pre-history (?): view interpolation Depth maps and images: view extrapolation [Matthies,Szeliski,Kanade 89] input depth image novel view View interpolation: warp and dissolve [Beier & Neely 92; Chen & Williams 93] Richard Szeliski Image-Based Rendering and Modeling 11
12 Panoramic image stitching Convert image sequence into a cylindrical image = Richard Szeliski Image-Based Rendering and Modeling 12
13 Panoramic image stitching Key technical breakthroughs: interactive viewing [(Lippman 80), Chen 95] general camera motions [1997 ] parallax compensation [1998 ] fully automated feature detection and matching [Brown & Lowe 03] seam selection [1975 (1998), 2001 ] Laplacian and Poisson blending [1984, 2003] Early example of computational photography Richard Szeliski Image-Based Rendering and Modeling 13
14 VR today: Gigapan Richard Szeliski Image-Based Rendering and Modeling 14
15 VR today: 360 cities Richard Szeliski Image-Based Rendering and Modeling 15
16 VR today: Photosynth Richard Szeliski Image-Based Rendering and Modeling 16
17 Moving beyond single (or linked) panoramas
18 Panoramic Video-Based Tours [Uyttendaele et al. 2004] 2D 3D Light Field (?)
19 Video-Based Walkthroughs Move camera along a rail ( dolly track ) and play back a 360 video Applications: Homes and architecture Outdoor locations (tourist destinations) Richard Szeliski Image-Based Rendering and Modeling 19
![Ladybug ] Richard](/docs-images/75/72430630/images/20-2.jpg "Szeliski Image-Based")
20 Surround video acquisition system OmniCam (six-camera head) [ Point Grey Ladybug ] Richard Szeliski Image-Based Rendering and Modeling 20
21 Acquisition platforms (then) Robotic cart Wearable Richard Szeliski Image-Based Rendering and Modeling 22
22 Acquisition platforms (today) Richard Szeliski Image-Based Rendering and Modeling 23
23 Acquisition platforms (future?) Richard Szeliski Image-Based Rendering and Modeling 24
24 Acquisition platforms (now) 10 camera panoramic rig One bubble every 2 meters Reproject into cube maps Richard Szeliski Image-Based Rendering and Modeling 25
25 Lightfields and Lumigraphs True 4D (with lots of slides from Michael Cohen)
26 Modeling light How do we generate new scenes and animations from existing ones? Classic 3D Vision + Graphics : take (lots of) pictures recover camera pose build 3D model extract texture maps / BRDFs synthesize new views tons of great work and demos as 3DIMPVT Richard Szeliski Image-Based Rendering and Modeling 27
27 Computer Vision Output Model Real Scene Real Cameras Richard Szeliski Image-Based Rendering and Modeling 28
28 Computer Graphics Output Image Synthetic Camera Model Richard Szeliski Image-Based Rendering and Modeling 29
29 Combined Output Image Synthetic Camera Model Real Cameras Real Scene Richard Szeliski Image-Based Rendering and Modeling 30
30 But, vision technology falls short Output Image Synthetic Camera Model Real Cameras Real Scene Richard Szeliski Image-Based Rendering and Modeling 31
31 and so does graphics. Output Image Synthetic Camera Model Real Cameras Real Scene Richard Szeliski Image-Based Rendering and Modeling 32
32 Image Based Rendering Output Image Synthetic Real Scene Images+Model Camera Real Cameras -or- Expensive Image Synthesis Richard Szeliski Image-Based Rendering and Modeling 33
33 Lumigraph / Lightfield Outside convex space Empty Stuff 4D Richard Szeliski Image-Based Rendering and Modeling 34
34 Lumigraph - Organization 2D position 2D position s u 2 plane parameterization Richard Szeliski Image-Based Rendering and Modeling 35
35 Lumigraph - Organization 2D position 2D position t s,t s,t u,v v u,v 2 plane parameterization Richard Szeliski Image-Based Rendering and Modeling 36 s u
36 Lumigraph - Rendering For each output pixel determine s,t,u,v either find closest discrete RGB interpolate near values s,t u,v Richard Szeliski Image-Based Rendering and Modeling 37
37 Lumigraph - Rendering For each output pixel determine s,t,u,v either use closest discrete RGB interpolate near values Richard Szeliski Image-Based Rendering and Modeling 38 s u
38 Lumigraph - Rendering Nearest closest s closest u draw it Blend 16 nearest quadrilinear interpolation u Richard Szeliski Image-Based Rendering and Modeling 39 s
39 Lumigraph - Rendering Depth Correction quadralinear interpolation new closest like focus u Richard Szeliski Image-Based Rendering and Modeling 40 s
40 Lumigraph Image Effects Image effects: parallax occlusion transparency highlights Richard Szeliski Image-Based Rendering and Modeling 43
41 Unstructured Lumigraph What if the images aren t sampled on a regular 2D grid? can still re-sample rays ray weighting becomes more complex [Buehler et al., SIGGRAPH 2000] Richard Szeliski Image-Based Rendering and Modeling 45
42 Surface Light Fields [Wood et al, SIGGRAPH 2000] Turn 4D parameterization around: every surface pt. Leverage coherence: compress radiance fn (BRDF * illumination) after rotation by n Richard Szeliski Image-Based Rendering and Modeling 46
![Surface Light Fields [Wood et al, SIGGRAPH 2000]](/docs-images/75/72430630/images/43-0.jpg "Richard Szeliski Image-Based Rendering and Modeling")
43 Surface Light Fields [Wood et al, SIGGRAPH 2000] Richard Szeliski Image-Based Rendering and Modeling 47
44 2.5D Representations Images (and panoramas) are 2D Lumigraph is 4D 3D 3D Lumigraph subsets (360 video tours) Concentric mosaics 2.5D Layered Depth Images Sprites with Depth (impostors) Layered (Virtual Viewpoint) Video View Dependent Surfaces (see Façade) Richard Szeliski Image-Based Rendering and Modeling 49
45 Layered Depth Image 2.5 D? Layered Depth Image Richard Szeliski Image-Based Rendering and Modeling 50
46 Layered Depth Image Rendering from LDI [Shade et al., SIGGRAPH 98] Incremental in LDI X and Y Guaranteed to be in back-to-front order Richard Szeliski Image-Based Rendering and Modeling 51
47 Sprites with Depth Represent scene as collection of cutouts with depth (planes + parallax) Render back to front with fwd/inverse warping [Shade et al., SIGGRAPH 98] Basis of Virtual Viewpoint Video [Zitnick et al. 2004] Richard Szeliski Image-Based Rendering and Modeling 53
48 Virtual Viewpoint (3D) Video [Zitnick et al., SIGGRAPH 2004]
49 Scenarios for 3D video Sports events, e.g., CMU s 30-camera EyeVision system at SuperBowl XXXV) Concert performances, plays, circus acts Games Instructional video, e.g., golf, skating, martial arts Interactive (Internet) video Richard Szeliski Image-Based Rendering and Modeling 55
50 3D video: Challenges Capture: record multiple synchronized camera streams Processing and representation: ensure photorealism Compression: exploit redundancy Rendering: provide interactive viewpoint control Richard Szeliski Image-Based Rendering and Modeling 56
51 cameras concentrators hard disks controlling laptop Richard Szeliski Image-Based Rendering and Modeling 58
52 Matching pixels Easy Richard Szeliski Image-Based Rendering and Modeling 59
53 Harder Richard Szeliski Image-Based Rendering and Modeling 60
54 Really Really Hard Occlusion Richard Szeliski Image-Based Rendering and Modeling 61
55 Matting Some pixels get influence for multiple surfaces. Background Surface Foreground Surface Close up of real image: Image Camera Multiple colors and depths at boundary pixels Richard Szeliski Image-Based Rendering and Modeling 62
56 Find matting information: 1. Find boundary strips using depth. 2. Within boundary strips compute the colors and depths of the foreground and background object. Background Foreground Strip Width Richard Szeliski Image-Based Rendering and Modeling 63
57 Why matting is important No Matting Matting Richard Szeliski Image-Based Rendering and Modeling 64
58 Representation Main Boundary Main Layer: Color Depth Strip Width Boundary Layer: Color Alpha Depth Richard Szeliski Image-Based Rendering and Modeling 65
59 Richard Szeliski Image-Based Rendering and Modeling 66
60 Outline Panoramas and 360 Video Walkthroughs Light Fields and Lumigraphs LDIs, Sprites, and Layered Video Image-Based Modeling Environment Mattes and Matting Photo Tourism and Photosynth Point-Based Rendering and NPR Reflections and Transparency [2012] Richard Szeliski Image-Based Rendering and Modeling 67
61 Image-Based Modeling
62 Image-Based Modeling Richard Szeliski Image-Based Rendering and Modeling 69
63 Façade 1. Select building blocks 2. Align them in each image 3. Solve for camera pose and block parameters (using constraints) Richard Szeliski Image-Based Rendering and Modeling 71
64 View-dependent texture mapping 1. Determine visible cameras for each surface element 2. Blend textures (images) depending on distance between original camera and novel viewpoint Richard Szeliski Image-Based Rendering and Modeling 72
65 Graz reconstruction Fully automated multi-view stereo, ca Richard Szeliski Image-Based Rendering and Modeling 74
66 Non-photorealistic experiences
![Szeliski, SIGGRAPH 2006] Richard](/docs-images/75/72430630/images/67-1.jpg "Szeliski Image-Based Rendering and")
67 Photo Tourism Images on the Internet Computed 3D structure [Snavely, Seitz, Szeliski, SIGGRAPH 2006] Richard Szeliski Image-Based Rendering and Modeling 76
68 Photo Tourism Richard Szeliski Image-Based Rendering and Modeling 77
69 Internet Computer Vision Use the Internet as an image and/or annotation source to solve challenging vision problems. Richard Szeliski Image-Based Rendering and Modeling 78
70 Internet Computer Vision Richard Szeliski Image-Based Rendering and Modeling 79
71 Internet Computer Vision Richard Szeliski Image-Based Rendering and Modeling 80
72 Recent research
73 Piecewise planar proxies [Sinha, Steedly, Szeliski ICCV 09] Richard Szeliski Image-Based Rendering and Modeling 82
74 Building Interiors [ Furukawa et al., ICCV 09] Richard Szeliski Image-Based Rendering and Modeling 84
75 Internet-Scale Stereo [ Furukawa et al., CVPR 10] Richard Szeliski Image-Based Rendering and Modeling 85
76 What s missing? Reflections Gloss? Richard Szeliski Image-Based Rendering and Modeling 86
77 Reflections Richard Szeliski Image-Based Rendering and Modeling 87
78 Image-Based Rendering for Scenes with Reflections Sudipta N. Sinha Johannes Kopf Michael Goesele Daniel Scharstein Richard Szeliski
79 Wrapping up
80 Graphics/Imaging Continuum Concentric mosaics Geometry centric Image centric Fixed geometry Viewdependent texture Viewdependent geometry Sprites with depth LDI Lumigraph Light field Polygon rendering + texture mapping Warping Interpolation Richard Szeliski Image-Based Rendering and Modeling 90
81 What works? What doesn t? Automatic 3D pose estimation Aerial (+ active ground level) modeling Accurate boundaries & matting Reflections and transparency User-generated content Casually acquired photos and videos New sensors Active sensing (Kinect ); Plenoptic cameras Integration with recognition Richard Szeliski Image-Based Rendering and Modeling 91
82 What about Computational Photography? Richard Szeliski Image-Based Rendering and Modeling 92
83 Image-Based Rendering Panoramas and 360 Video Walkthroughs Light Fields and Lumigraphs Layered Representations and Matting Image-Based Modeling Environment Mattes and Matting Photo Tourism and Photosynth Point-Based Rendering and NPR Reflections and Transparency Where next? How to use in your applications? Richard Szeliski Image-Based Rendering and Modeling 93
84 Graphics/Imaging Continuum Concentric mosaics Geometry centric Image centric Fixed geometry Viewdependent texture Viewdependent geometry Sprites with depth LDI Lumigraph Light field Polygon rendering + texture mapping Warping Interpolation Richard Szeliski Image-Based Rendering and Modeling 94
85 ( Questions ) [ The End ]
Image or Object? Is this real?
Image or Object? Michael F. Cohen Microsoft Is this real? Photo by Patrick Jennings (patrick@synaptic.bc.ca), Copyright 1995, 96, 97 Whistler B. C. Canada Modeling, Rendering, and Lighting 1 A mental model?
But, vision technology falls short. and so does graphics. Image Based Rendering. Ray. Constant radiance. time is fixed. 3D position 2D direction
Computer Graphics -based rendering Output Michael F. Cohen Microsoft Research Synthetic Camera Model Computer Vision Combined Output Output Model Real Scene Synthetic Camera Model Real Cameras Real Scene
Image-Based Rendering
Image-Based Rendering COS 526, Fall 2016 Thomas Funkhouser Acknowledgments: Dan Aliaga, Marc Levoy, Szymon Rusinkiewicz What is Image-Based Rendering? Definition 1: the use of photographic imagery to overcome
Modeling Light. Michal Havlik : Computational Photography Alexei Efros, CMU, Fall 2007
Modeling Light Michal Havlik 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 The Plenoptic Function Figure by Leonard McMillan Q: What is the set of all things that we can ever see? A: The
A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India
A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India Keshav Mahavidyalaya, University of Delhi, Delhi, India Abstract
Image-Based Modeling and Rendering
Traditional Computer Graphics Image-Based Modeling and Rendering Thomas Funkhouser Princeton University COS 426 Guest Lecture Spring 2003 How would you model and render this scene? (Jensen) How about this
Modeling Light. Slides from Alexei A. Efros and others
Project 3 Results http://www.cs.brown.edu/courses/cs129/results/proj3/jcmace/ http://www.cs.brown.edu/courses/cs129/results/proj3/damoreno/ http://www.cs.brown.edu/courses/cs129/results/proj3/taox/ Stereo
The Light Field and Image-Based Rendering
Lecture 11: The Light Field and Image-Based Rendering Visual Computing Systems Demo (movie) Royal Palace: Madrid, Spain Image-based rendering (IBR) So far in course: rendering = synthesizing an image from
A million pixels, a million polygons. Which is heavier? François X. Sillion. imagis* Grenoble, France
A million pixels, a million polygons. Which is heavier? François X. Sillion imagis* Grenoble, France *A joint research project of CNRS, INRIA, INPG and UJF MAGIS Why this question? Evolution of processing
Modeling Light. Michal Havlik
Modeling Light Michal Havlik 15-463: Computational Photography Alexei Efros, CMU, Spring 2010 What is light? Electromagnetic radiation (EMR) moving along rays in space R( ) is EMR, measured in units of
Lecture 15: Image-Based Rendering and the Light Field. Kayvon Fatahalian CMU : Graphics and Imaging Architectures (Fall 2011)
Lecture 15: Image-Based Rendering and the Light Field Kayvon Fatahalian CMU 15-869: Graphics and Imaging Architectures (Fall 2011) Demo (movie) Royal Palace: Madrid, Spain Image-based rendering (IBR) So
Image-Based Modeling and Rendering. Image-Based Modeling and Rendering. Final projects IBMR. What we have learnt so far. What IBMR is about
Image-Based Modeling and Rendering Image-Based Modeling and Rendering MIT EECS 6.837 Frédo Durand and Seth Teller 1 Some slides courtesy of Leonard McMillan, Wojciech Matusik, Byong Mok Oh, Max Chen 2
Image Base Rendering: An Introduction
Image Base Rendering: An Introduction Cliff Lindsay CS563 Spring 03, WPI 1. Introduction Up to this point, we have focused on showing 3D objects in the form of polygons. This is not the only approach to
Image-Based Rendering. Johns Hopkins Department of Computer Science Course : Rendering Techniques, Professor: Jonathan Cohen
Image-Based Rendering Image-Based Rendering What is it? Still a difficult question to answer Uses images (photometric( info) as key component of model representation What s Good about IBR Model acquisition
Light Field Spring
Light Field 2015 Spring Recall: Light is Electromagnetic radiation (EMR) moving along rays in space R(l) is EMR, measured in units of power (watts) l is wavelength Useful things: Light travels in straight
Algorithms for Image-Based Rendering with an Application to Driving Simulation
Algorithms for Image-Based Rendering with an Application to Driving Simulation George Drettakis GRAPHDECO/Inria Sophia Antipolis, Université Côte d Azur http://team.inria.fr/graphdeco Graphics for Driving
Modeling Light. Michal Havlik : Computational Photography Alexei Efros, CMU, Fall 2011
Modeling Light Michal Havlik 15-463: Computational Photography Alexei Efros, CMU, Fall 2011 What is light? Electromagnetic radiation (EMR) moving along rays in space R(λ) is EMR, measured in units of power
Real-time Generation and Presentation of View-dependent Binocular Stereo Images Using a Sequence of Omnidirectional Images
Real-time Generation and Presentation of View-dependent Binocular Stereo Images Using a Sequence of Omnidirectional Images Abstract This paper presents a new method to generate and present arbitrarily
Image-based modeling (IBM) and image-based rendering (IBR)
Image-based modeling (IBM) and image-based rendering (IBR) CS 248 - Introduction to Computer Graphics Autumn quarter, 2005 Slides for December 8 lecture The graphics pipeline modeling animation rendering
More and More on Light Fields. Last Lecture
More and More on Light Fields Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 4 Last Lecture Re-review with emphasis on radiometry Mosaics & Quicktime VR The Plenoptic function The main
Morphable 3D-Mosaics: a Hybrid Framework for Photorealistic Walkthroughs of Large Natural Environments
Morphable 3D-Mosaics: a Hybrid Framework for Photorealistic Walkthroughs of Large Natural Environments Nikos Komodakis and Georgios Tziritas Computer Science Department, University of Crete E-mails: {komod,
Modeling Light. On Simulating the Visual Experience
Modeling Light 15-463: Rendering and Image Processing Alexei Efros On Simulating the Visual Experience Just feed the eyes the right data No one will know the difference! Philosophy: Ancient question: Does
Volumetric Scene Reconstruction from Multiple Views
Volumetric Scene Reconstruction from Multiple Views Chuck Dyer University of Wisconsin dyer@cs cs.wisc.edu www.cs cs.wisc.edu/~dyer Image-Based Scene Reconstruction Goal Automatic construction of photo-realistic
Photo Tourism: Exploring Photo Collections in 3D
Photo Tourism: Exploring Photo Collections in 3D SIGGRAPH 2006 Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research 2006 2006 Noah Snavely Noah Snavely Reproduced with
Homographies and RANSAC
Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position
A million pixels, a million polygons: which is heavier?
A million pixels, a million polygons: which is heavier? François X. Sillion To cite this version: François X. Sillion. A million pixels, a million polygons: which is heavier?. Eurographics 97, Sep 1997,
EE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry
Jingyi Yu CISC 849. Department of Computer and Information Science
Digital Photography and Videos Jingyi Yu CISC 849 Light Fields, Lumigraph, and Image-based Rendering Pinhole Camera A camera captures a set of rays A pinhole camera captures a set of rays passing through
Multiple View Geometry
Multiple View Geometry Martin Quinn with a lot of slides stolen from Steve Seitz and Jianbo Shi 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Our Goal The Plenoptic Function P(θ,φ,λ,t,V
Modeling Light. Michal Havlik
Modeling Light Michal Havlik 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 What is light? Electromagnetic radiation (EMR) moving along rays in space R(λ) is EMR, measured in units of power
Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration
An Algorithm for Seamless Image Stitching and Its Application
An Algorithm for Seamless Image Stitching and Its Application Jing Xing, Zhenjiang Miao, and Jing Chen Institute of Information Science, Beijing JiaoTong University, Beijing 100044, P.R. China Abstract.
Real Time Rendering. CS 563 Advanced Topics in Computer Graphics. Songxiang Gu Jan, 31, 2005
Real Time Rendering CS 563 Advanced Topics in Computer Graphics Songxiang Gu Jan, 31, 2005 Introduction Polygon based rendering Phong modeling Texture mapping Opengl, Directx Point based rendering VTK
Image warping and stitching
Image warping and stitching Thurs Oct 15 Last time Feature-based alignment 2D transformations Affine fit RANSAC 1 Robust feature-based alignment Extract features Compute putative matches Loop: Hypothesize
Layered Depth Panoramas
Layered Depth Panoramas Ke Colin Zheng 1, Sing Bing Kang 2 Michael F. Cohen 2, Richard Szeliski 2 1 University of Washington, Seattle, WA 2 Microsoft Research, Redmond, WA Abstract Representations for
Computational Photography
End of Semester is the last lecture of new material Quiz on Friday 4/30 Sample problems are posted on website Computational Photography Final Project Presentations Wednesday May 12 1-5pm, CII 4040 Attendance
Image-Based Rendering. Johns Hopkins Department of Computer Science Course : Rendering Techniques, Professor: Jonathan Cohen
Image-Based Rendering Image-Based Rendering What is it? Still a difficult question to answer Uses images (photometric( info) as key component of model representation What s Good about IBR Model acquisition
Hybrid Rendering for Collaborative, Immersive Virtual Environments
Hybrid Rendering for Collaborative, Immersive Virtual Environments Stephan Würmlin wuermlin@inf.ethz.ch Outline! Rendering techniques GBR, IBR and HR! From images to models! Novel view generation! Putting
Image-Based Rendering. Image-Based Rendering
Image-Based Rendering Image-Based Rendering What is it? Still a difficult question to answer Uses images (photometric info) as key component of model representation 1 What s Good about IBR Model acquisition
Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2010 Today Light fields Introduction Light fields Signal processing analysis Light field cameras Application Introduction Pinhole camera
Processing 3D Surface Data
Processing 3D Surface Data Computer Animation and Visualisation Lecture 17 Institute for Perception, Action & Behaviour School of Informatics 3D Surfaces 1 3D surface data... where from? Iso-surfacing
Towards a Perceptual Method of Blending for Image-Based Models
Towards a Perceptual Method of Blending for Image-Based Models Gordon Watson, Patrick O Brien and Mark Wright Edinburgh Virtual Environment Centre University of Edinburgh JCMB, Mayfield Road, Edinburgh
CS 354R: Computer Game Technology
CS 354R: Computer Game Technology Texture and Environment Maps Fall 2018 Texture Mapping Problem: colors, normals, etc. are only specified at vertices How do we add detail between vertices without incurring
Capturing and View-Dependent Rendering of Billboard Models
Capturing and View-Dependent Rendering of Billboard Models Oliver Le, Anusheel Bhushan, Pablo Diaz-Gutierrez and M. Gopi Computer Graphics Lab University of California, Irvine Abstract. In this paper,
What have we leaned so far?
What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging What have we learned so far? Image Filtering Image Warping Camera Projection Model Project 2: Panoramic
Image stitching. Announcements. Outline. Image stitching
Announcements Image stitching Project #1 was due yesterday. Project #2 handout will be available on the web later tomorrow. I will set up a webpage for artifact voting soon. Digital Visual Effects, Spring
Announcements. Mosaics. How to do it? Image Mosaics
Announcements Mosaics Project artifact voting Project 2 out today (help session at end of class) http://www.destination36.com/start.htm http://www.vrseattle.com/html/vrview.php?cat_id=&vrs_id=vrs38 Today
More Single View Geometry
More Single View Geometry with a lot of slides stolen from Steve Seitz Cyclops Odilon Redon 1904 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Final Projects Are coming up fast! Undergrads
Image Based Rendering. D.A. Forsyth, with slides from John Hart
Image Based Rendering D.A. Forsyth, with slides from John Hart Topics Mosaics translating cameras reveal extra information, break occlusion Optical flow for very small movements of the camera Explicit
CSc Topics in Computer Graphics 3D Photography
CSc 83010 Topics in Computer Graphics 3D Photography Tuesdays 11:45-1:45 1:45 Room 3305 Ioannis Stamos istamos@hunter.cuny.edu Office: 1090F, Hunter North (Entrance at 69 th bw/ / Park and Lexington Avenues)
Image-Based Rendering and Light Fields
CS194-13: Advanced Computer Graphics Lecture #9 Image-Based Rendering University of California Berkeley Image-Based Rendering and Light Fields Lecture #9: Wednesday, September 30th 2009 Lecturer: Ravi
Evolution of Imaging Technology in Computer Graphics. Related Areas
Evolution of Imaging Technology in Computer Graphics Jonas Gomes Rio de Janeiro http://www.visgraf.impa.br Image computation Related Areas 1 Related Areas An integrated view Problems on the frontiers Graphical
Projective Texture Mapping with Full Panorama
EUROGRAPHICS 2002 / G. Drettakis and H.-P. Seidel Volume 21 (2002), Number 3 (Guest Editors) Projective Texture Mapping with Full Panorama Dongho Kim and James K. Hahn Department of Computer Science, The
Mosaics. Today s Readings
Mosaics VR Seattle: http://www.vrseattle.com/ Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html Today s Readings Szeliski and Shum paper (sections
Image Morphing. Application: Movie Special Effects. Application: Registration /Alignment. Image Cross-Dissolve
Image Morphing Application: Movie Special Effects Morphing is turning one image into another (through a seamless transition) First movies with morphing Willow, 1988 Indiana Jones and the Last Crusade,
CS 684 Fall 2005 Image-based Modeling and Rendering. Ruigang Yang
CS 684 Fall 2005 Image-based Modeling and Rendering Ruigang Yang Administrivia Classes: Monday and Wednesday, 4:00-5:15 PM Instructor: Ruigang Yang ryang@cs.uky.edu Office Hour: Robotics 514D, MW 1500-1600
Dense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
Image Warping and Mosacing
Image Warping and Mosacing 15-463: Rendering and Image Processing Alexei Efros with a lot of slides stolen from Steve Seitz and Rick Szeliski Today Mosacs Image Warping Homographies Programming Assignment
Multi-view stereo. Many slides adapted from S. Seitz
Multi-view stereo Many slides adapted from S. Seitz Beyond two-view stereo The third eye can be used for verification Multiple-baseline stereo Pick a reference image, and slide the corresponding window
Computer Vision. I-Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung University
Computer Vision I-Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung University About the course Course title: Computer Vision Lectures: EC016, 10:10~12:00(Tues.); 15:30~16:20(Thurs.) Pre-requisites:
Noah Snavely Steven M. Seitz. Richard Szeliski. University of Washington. Microsoft Research. Modified from authors slides
Photo Tourism: Exploring Photo Collections in 3D Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research 2006 2006 Noah Snavely Noah Snavely Modified from authors slides
Processing 3D Surface Data
Processing 3D Surface Data Computer Animation and Visualisation Lecture 12 Institute for Perception, Action & Behaviour School of Informatics 3D Surfaces 1 3D surface data... where from? Iso-surfacing
Recap. DoF Constraint Solver. translation. affine. homography. 3D rotation
Image Blending Recap DoF Constraint Solver translation affine homography 3D rotation Recap DoF Constraint Solver translation 2 affine homography 3D rotation Recap DoF Constraint Solver translation 2 affine
Today s lecture. Image Alignment and Stitching. Readings. Motion models
Today s lecture Image Alignment and Stitching Computer Vision CSE576, Spring 2005 Richard Szeliski Image alignment and stitching motion models cylindrical and spherical warping point-based alignment global
Image warping and stitching
Image warping and stitching May 4 th, 2017 Yong Jae Lee UC Davis Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 2 Alignment problem In alignment, we will
Computational Photography: Advanced Topics. Paul Debevec
Computational Photography: Advanced Topics Paul Debevec Class: Computational Photography, Advanced Topics Module 1: 105 minutes Debevec,, Raskar and Tumblin 1:45: A.1 Introduction and Overview (Raskar,
On the Use of Ray-tracing for Viewpoint Interpolation in Panoramic Imagery
On the Use of Ray-tracing for Viewpoint Interpolation in Panoramic Imagery Feng Shi, Robert Laganière, Eric Dubois VIVA research lab SITE, University of Ottawa Ottawa, ON, Canada, K1N 6N5 {fshi098,laganier,edubois}@site.uottawa.ca
Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration
Video Mosaics for Virtual Environments, R. Szeliski. Review by: Christopher Rasmussen
Video Mosaics for Virtual Environments, R. Szeliski Review by: Christopher Rasmussen September 19, 2002 Announcements Homework due by midnight Next homework will be assigned Tuesday, due following Tuesday.
Dense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction
3-D Shape Reconstruction from Light Fields Using Voxel Back-Projection
3-D Shape Reconstruction from Light Fields Using Voxel Back-Projection Peter Eisert, Eckehard Steinbach, and Bernd Girod Telecommunications Laboratory, University of Erlangen-Nuremberg Cauerstrasse 7,
Multi-view Stereo. Ivo Boyadzhiev CS7670: September 13, 2011
Multi-view Stereo Ivo Boyadzhiev CS7670: September 13, 2011 What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape
Image-Based Rendering using Image-Warping Motivation and Background
Image-Based Rendering using Image-Warping Motivation and Background Leonard McMillan LCS Computer Graphics Group MIT The field of three-dimensional computer graphics has long focused on the problem of
Scene Modeling for a Single View
Scene Modeling for a Single View René MAGRITTE Portrait d'edward James with a lot of slides stolen from Steve Seitz and David Brogan, 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Classes
Plenoptic Image Editing
Plenoptic Image Editing Steven M. Seitz Computer Sciences Department University of Wisconsin Madison Madison, WI53706 seitz@cs.wisc.edu Kiriakos N. Kutulakos Department of Computer Science University of
Multi-View Stereo for Static and Dynamic Scenes
Multi-View Stereo for Static and Dynamic Scenes Wolfgang Burgard Jan 6, 2010 Main references Yasutaka Furukawa and Jean Ponce, Accurate, Dense and Robust Multi-View Stereopsis, 2007 C.L. Zitnick, S.B.
Image warping and stitching
Image warping and stitching May 5 th, 2015 Yong Jae Lee UC Davis PS2 due next Friday Announcements 2 Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 3 Alignment
A FREE-VIEWPOINT VIDEO SYSTEM FOR VISUALISATION OF SPORT SCENES
A FREE-VIEWPOINT VIDEO SYSTEM FOR VISUALISATION OF SPORT SCENES Oliver Grau 1, Adrian Hilton 2, Joe Kilner 2, Gregor Miller 2 Tim Sargeant 1, Jonathan Starck 2 1 BBC Research, UK Oliver.Grau Tim.Sargeant
Announcements. Mosaics. Image Mosaics. How to do it? Basic Procedure Take a sequence of images from the same position =
Announcements Project 2 out today panorama signup help session at end of class Today mosaic recap blending Mosaics Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html
Multiview Reconstruction
Multiview Reconstruction Why More Than 2 Views? Baseline Too short low accuracy Too long matching becomes hard Why More Than 2 Views? Ambiguity with 2 views Camera 1 Camera 2 Camera 3 Trinocular Stereo
Chaplin, Modern Times, 1936
Chaplin, Modern Times, 1936 [A Bucket of Water and a Glass Matte: Special Effects in Modern Times; bonus feature on The Criterion Collection set] Multi-view geometry problems Structure: Given projections
Light Fields. Johns Hopkins Department of Computer Science Course : Rendering Techniques, Professor: Jonathan Cohen
Light Fields Light Fields By Levoy and Hanrahan, SIGGRAPH 96 Representation for sampled plenoptic function stores data about visible light at various positions and directions Created from set of images
3D Editing System for Captured Real Scenes
3D Editing System for Captured Real Scenes Inwoo Ha, Yong Beom Lee and James D.K. Kim Samsung Advanced Institute of Technology, Youngin, South Korea E-mail: {iw.ha, leey, jamesdk.kim}@samsung.com Tel:
View-Dependent Texture Mapping
81 Chapter 6 View-Dependent Texture Mapping 6.1 Motivation Once a model of an architectural scene is recovered, the goal is to produce photorealistic renderings. A traditional approach that is consistent
Image Transfer Methods. Satya Prakash Mallick Jan 28 th, 2003
Image Transfer Methods Satya Prakash Mallick Jan 28 th, 2003 Objective Given two or more images of the same scene, the objective is to synthesize a novel view of the scene from a view point where there
Photo Tourism: Exploring Photo Collections in 3D
Photo Tourism: Exploring Photo Collections in 3D Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research 15,464 37,383 76,389 2006 Noah Snavely 15,464 37,383 76,389 Reproduced
Image-Based Deformation of Objects in Real Scenes
Image-Based Deformation of Objects in Real Scenes Han-Vit Chung and In-Kwon Lee Dept. of Computer Science, Yonsei University sharpguy@cs.yonsei.ac.kr, iklee@yonsei.ac.kr Abstract. We present a new method
What is Computer Vision? Introduction. We all make mistakes. Why is this hard? What was happening. What do you see? Intro Computer Vision
What is Computer Vision? Trucco and Verri (Text): Computing properties of the 3-D world from one or more digital images Introduction Introduction to Computer Vision CSE 152 Lecture 1 Sockman and Shapiro:
CS6670: Computer Vision
CS6670: Computer Vision Noah Snavely Lecture 7: Image Alignment and Panoramas What s inside your fridge? http://www.cs.washington.edu/education/courses/cse590ss/01wi/ Projection matrix intrinsics projection
Multi-View Stereo for Community Photo Collections Michael Goesele, et al, ICCV Venus de Milo
Vision Sensing Multi-View Stereo for Community Photo Collections Michael Goesele, et al, ICCV 2007 Venus de Milo The Digital Michelangelo Project, Stanford How to sense 3D very accurately? How to sense
CS5670: Computer Vision
CS5670: Computer Vision Noah Snavely Multi-view stereo Announcements Project 3 ( Autostitch ) due Monday 4/17 by 11:59pm Recommended Reading Szeliski Chapter 11.6 Multi-View Stereo: A Tutorial Furukawa
Image-Based Rendering and Modeling. IBR Approaches for View Synthesis
Image-Based Rendering and Modeling l Image-based rendering (IBR): A scene is represented as a collection of images l 3D model-based rendering (MBR): A scene is represented by a 3D model plus texture maps
Computer Graphics. Lecture 9 Environment mapping, Mirroring
Computer Graphics Lecture 9 Environment mapping, Mirroring Today Environment Mapping Introduction Cubic mapping Sphere mapping refractive mapping Mirroring Introduction reflection first stencil buffer
Miniature faking. In close-up photo, the depth of field is limited.
Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg
Image Based Rendering
Image Based Rendering an overview Photographs We have tools that acquire and tools that display photographs at a convincing quality level 2 1 3 4 2 5 6 3 7 8 4 9 10 5 Photographs We have tools that acquire
Augmented and Mixed Reality
Augmented and Mixed Reality Uma Mudenagudi Dept. of Computer Science and Engineering, Indian Institute of Technology Delhi Outline Introduction to Augmented Reality(AR) and Mixed Reality(MR) A Typical
Local Feature Detectors
Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,
3D Modeling of Objects Using Laser Scanning
1 3D Modeling of Objects Using Laser Scanning D. Jaya Deepu, LPU University, Punjab, India Email: Jaideepudadi@gmail.com Abstract: In the last few decades, constructing accurate three-dimensional models
Today. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry
Neue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen?
Neue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen? Fraunhofer HHI 13.07.2017 1 Fraunhofer-Gesellschaft Fraunhofer is Europe s largest organization for applied research.