Algorithms for Image-Based Rendering with an Application to Driving Simulation
|
|
|
- Felicity Thornton
- 5 years ago
- Views:
Transcription
1 Algorithms for Image-Based Rendering with an Application to Driving Simulation George Drettakis GRAPHDECO/Inria Sophia Antipolis, Université Côte d Azur
2 Graphics for Driving Simulators Requirements for Computer Graphics (CG) in driving simulators Realism Large environments Ease of Creation Variety Current practice Computer Graphics with Synthetic Content
3 Current Graphics for Simulators Getting better but: Hard to create environments Realism not quite there yet CARLA Intel Gazebo
4 Computer Graphics (CG) with Synthetic Content Can provide realistic content, but Requires huge amount of manual labor, high cost Rendering quality can be good, but expensive CG is about creating images Ultimately a forward process Often involves simulation or procedural methods
5 Captured Content Multi view stereo, game changer Multiplication of sensors Traditional expensive: LIDAR etc. Cheaper, accessible: Kinect etc Capture methods seen as inverse problem Frequently use optimization methods Image synthesis is often the ultimate goal
6 Traditional Computer Graphics 3D geometry projection Simulation physics
7 The richness of our everyday world Photo by Svetlana Lazebnik
8 Creating Realistic Imagery Computer Graphics Photography + great creative possibilities + easy to manipulate objects / viewpoint / light - Tremendous expertise and work for realism + instantly realistic + easy to acquire - very hard to manipulate objects / viewpoint / light
9 Creating Realistic Imagery Computer Graphics Image Based Render (IBR) Photography Realism Ease of creation Manipulation + great creative possibilities + easy to manipulate objects / viewpoint / light - Tremendous expertise and work for realism + instantly realistic + easy to acquire - very hard to manipulate objects / viewpoint / light
10 Image synthesis Output Image Virtual Camera Virtual scene
11 Computer vision Output Virtual scene Real scene Real cameras
12 Computer vision and graphics Output Image Virtual camera Virtual scene Real cameras Real scene
13 Imperfect computer vision Output Image Virtual camera Virtual scene Real cameras Real scene
14 Imperfect computer graphics Output Image Virtual camera Virtual scene
15 Image-based rendering (IBR) Image Virtual camera Images + Virtual scene Real cameras Real scene
16 Image-based rendering Blend input images Virtual Novel camera Use Real proxy scene geometry Real Input cameras
17 Background on IBR
![Capturing multiple viewpoints Capture the Lightfield [Levoy and Hanrahan, SIGGRAPH 1996] Every light ray](/docs-images/86/93466258/images/18-0.jpg "from camera plane (u,v) in direction of the image plane (s,t) Lots of data! High redundancy, compression")
18 Capturing multiple viewpoints Capture the Lightfield [Levoy and Hanrahan, SIGGRAPH 1996] Every light ray from camera plane (u,v) in direction of the image plane (s,t) Lots of data! High redundancy, compression
19 View-dependent texture mapping [Debevec & Malik SIGGRAPH 1997]
20 View-dependent texture mapping [Debevec & Malik SIGGRAPH 1997]
21 Unstructured Lumigraph (ULR) [Buelher et al. SIGGRAPH 2001] 1ST REVIEW MEETING Sophia Antipolis 25/26 November
22 Recent Image-Based Rendering Approximate multi view capture of a scene Camera calibration (Structure from Motion, SfM) Multi view Stereo reconstruction (MVS) Now a mature technology, commercially available solutions Bentley/Acute3D ContextCapture RealityCapture PhotoScan etc
23 Multi-View Stereo
24 Multi-View Stereo
25 SfM and MVS We have the positions and orientations of the camera for each photo We have an approximate geometric model Missing parts and details Overestimated geometry Bad normals/surface reconstruction
26 How to render?
27 Option 1: Textured Mesh
28 Textured Mesh
29 Option 2: Traditional IBR Basic Idea Use MVS geometry and blend images using a set of clever weights Image blending: Corrects some of the geometry errors Provides some view dependent effects 1ST REVIEW MEETING Sophia Antipolis 25/26 November
30 Option 2: ULR [Buehler 2001]
31 Prior work: ULR? [Buehler2001] Moving highlights
32 Novel Solutions for IBR at Inria/GRAPHDECO Address two main shortcomings: Inaccurate proxy geometry Finding good blending weights How? Create a representation per view overcoming limitations on geometry New algorithms to improve blending of different views 1ST REVIEW MEETING Sophia Antipolis 25/26 November
33 Depth synthesis and local warps for plausible image-based navigation G. Chaurasia, S. Duchêne, O. Sorkine Hornung, G. Drettakis ACM Trans. On Graphics ST REVIEW MEETING Sophia Antipolis 25/26 November
34 Main Problems Sparse depth Occlusion boundaries??
35 Compensate using 2D image-based approximations Over Depth Image Adaptive segmentation synthesis warping blending Fill depth in empty Alleviate regions distortions Blend Recover - selectively not and photoconsistent, occlusion cracks avoid boundaries due ghosting only inaccurate suitable depth for image warps
36 Local Shape Preserving Warp Image reprojected into novel view using depth samples Image warped into novel view using shape preserving warp
37 Warp and Blend Warped images Blended result
38 Results
39 Scalable Inside-Out Image-Based Rendering Peter Hedman 1, Tobias Ritschel 1, George Drettakis 2, Gabriel Brostow 1 UCL 1, Inria 2 ACM Transactions on Graphics (SIGGRAPH Asia 2016)
40 Improving Per-View Representation Improve per view representation Per view meshes Start with Indoors Scenes Initial version using depth sensors
41 FINAL REVIEW MEETING Darmstadt 29th November 2016 Indoors Scenes
42 Indoors Scene Use Kinect to get reasonable global geometry Needs a large number of photos to cover the space Larger overlap than outdoors FINAL REVIEW MEETING Darmstadt 29th November 2016
43 Prior work: Unstructured Lumigraph (ULR)? [Devec1998, Buehler2001, Eisemann2008] Inaccurate geometry
44 Prior work: Per-photograph geometry! [Ours, Zitnick2007, Chaurasia2013, Kopf2013, Ortiz-Cayon2015] Refined geometry
45 Refining per-photograph geometry [Ma2013, Wolff2016] Post process filters Global geometry Multi-view stereo refinement Global geometry prior Triangulated mesh
46 Indoors scenes
47 Indoors scenes
48 Deep Blending for Image Based Rendering Peter Hedman 1, Julien Philip 3, True Price 2,Jan Michael Frahm 2, George Drettakis 3, Gabriel Brostow 1 UCL 1, UNC 2, Inria 3 ACM Transactions on Graphics (SIGGRAPH Asia 2018 conditionally accepted) 1ST REVIEW MEETING Sophia Antipolis 25/26 November
49 Deep Blending Further improve per view meshes Better refinement Edge aware mesh simplification Can treat outdoors scenes! Learn blend weights by training in a leave one out manner Train on 19 datasets, total of 2600 images Learn blending weights 1ST REVIEW MEETING Sophia Antipolis 25/26 November
50 Deep Blending Results 1ST REVIEW MEETING Sophia Antipolis 25/26 November
51 Outdoors Scene: texture mapping 1ST REVIEW MEETING Sophia Antipolis 25/26 November
52 Deep Blending Results 1ST REVIEW MEETING Sophia Antipolis 25/26 November
53 Additional Challenges Changing the scene is hard in IBR Two advances: Removing / moving objects Changing the lighting ( relighting ) 1ST REVIEW MEETING Sophia Antipolis 25/26 November
54 Plane-Based Multi-View Inpainting for Image-Based Rendering Julien Philip Inria, Université Côte d Azur George Drettakis Inria, Université Côte d Azur ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games Montreal, Quebec, Canada: 17 May 2018 DEMO
55 Remove Objects Perform multi view inpainting Estimate local planar approximation of surface Perform efficient multi scale, resolution aware inpainting using PatchMatch Inpaint planar regions Inpaint input images
56 Some of the Input Images
57 Our result Our result
58 Our Our result result
59 Multi-View Relighting for Outdoors Scenes S. Duchêne, C. Riant, G. Chaurasia, J. Lopez-Moreno, P- Y. Laffont, S. Popov, A. Bousseau, G. Drettakis ACM Transactions on Graphics 2015
60 Intrinsic Decomposition Guide decomposition combining 3D information, rendering and image processing Reflectance Input Image Shading
61 Results
62 Image-Based Relighting
63 Application to Image-Based Rendering with Relighting
64 Driving Simulation Application First challenge: capture of large scenes Previous examples images Need images for a city block Solution: Inria bike capture: 1ST REVIEW MEETING Sophia Antipolis 25/26 November
65 Driving Simulation Application Second challenge: Rendering large scenes with thousands of pictures CPU memory; loading from disk GPU memory; loading from CPU (Initial) Solution: Streaming system 1ST REVIEW MEETING Sophia Antipolis 25/26 November
66 Driving Simulation Application Demo! Note: Semi automatic cleanup for mesh outliers and moving cars (in the process of being automated) 1ST REVIEW MEETING Sophia Antipolis 25/26 November
67 1ST REVIEW MEETING Sophia Antipolis 25/26 November
68 Future Work Correct remaining artifacts Solution for reconstruction & rendering of cars Hard cases in general: Thin structures, Reflections, transparency Ad hoc, specific solutions Semantics & Learning to the rescue? 1ST REVIEW MEETING Sophia Antipolis 25/26 November
69 Future Work Allow more editing Better relighting solution More geometry editing Seamless mix with synthetic content Combine rendering modalities; develop solutions that are not mutually exclusive (ERC FUNGRAPH) 1ST REVIEW MEETING Sophia Antipolis 25/26 November
70 Conclusions Very promising technology to provide highly realistic CG for driving simulators But technology must mature before prime time Could be a game changer for realism in driving simulation (Human) Training in assisted driving (Machine) Training for autonomous driving 1ST REVIEW MEETING Sophia Antipolis 25/26 November
71 Funding EU FP7 VERVE EU FP7 CR PLAY play.eu/ EU H2020 EMOTIVE ANR SEMAPOLIS 1ST REVIEW MEETING Sophia Antipolis 25/26 November
72 Thank you Questions?
Image-Based Modeling and Rendering
Image-Based Modeling and Rendering Richard Szeliski Microsoft Research IPAM Graduate Summer School: Computer Vision July 26, 2013 How far have we come? Light Fields / Lumigraph - 1996 Richard Szeliski
Image-Based Rendering
Image-Based Rendering COS 526, Fall 2016 Thomas Funkhouser Acknowledgments: Dan Aliaga, Marc Levoy, Szymon Rusinkiewicz What is Image-Based Rendering? Definition 1: the use of photographic imagery to overcome
The Light Field and Image-Based Rendering
Lecture 11: The Light Field and Image-Based Rendering Visual Computing Systems Demo (movie) Royal Palace: Madrid, Spain Image-based rendering (IBR) So far in course: rendering = synthesizing an image from
3D Editing System for Captured Real Scenes
3D Editing System for Captured Real Scenes Inwoo Ha, Yong Beom Lee and James D.K. Kim Samsung Advanced Institute of Technology, Youngin, South Korea E-mail: {iw.ha, leey, jamesdk.kim}@samsung.com Tel:
Lecture 15: Image-Based Rendering and the Light Field. Kayvon Fatahalian CMU : Graphics and Imaging Architectures (Fall 2011)
Lecture 15: Image-Based Rendering and the Light Field Kayvon Fatahalian CMU 15-869: Graphics and Imaging Architectures (Fall 2011) Demo (movie) Royal Palace: Madrid, Spain Image-based rendering (IBR) So
Capturing and View-Dependent Rendering of Billboard Models
Capturing and View-Dependent Rendering of Billboard Models Oliver Le, Anusheel Bhushan, Pablo Diaz-Gutierrez and M. Gopi Computer Graphics Lab University of California, Irvine Abstract. In this paper,
o Basic signal processing o Filtering, resampling, warping,... Rendering o Polygon rendering pipeline o Ray tracing Modeling
Background COS526: Advanced Computer Graphics Tom Funkhouser Fall 2010 Image Processing o Basic signal processing o Filtering, resampling, warping,... Rendering o Polygon rendering pipeline o Ray tracing
Morphable 3D-Mosaics: a Hybrid Framework for Photorealistic Walkthroughs of Large Natural Environments
Morphable 3D-Mosaics: a Hybrid Framework for Photorealistic Walkthroughs of Large Natural Environments Nikos Komodakis and Georgios Tziritas Computer Science Department, University of Crete E-mails: {komod,
But, vision technology falls short. and so does graphics. Image Based Rendering. Ray. Constant radiance. time is fixed. 3D position 2D direction
Computer Graphics -based rendering Output Michael F. Cohen Microsoft Research Synthetic Camera Model Computer Vision Combined Output Output Model Real Scene Synthetic Camera Model Real Cameras Real Scene
Image-Based Modeling and Rendering. Image-Based Modeling and Rendering. Final projects IBMR. What we have learnt so far. What IBMR is about
Image-Based Modeling and Rendering Image-Based Modeling and Rendering MIT EECS 6.837 Frédo Durand and Seth Teller 1 Some slides courtesy of Leonard McMillan, Wojciech Matusik, Byong Mok Oh, Max Chen 2
VIDEO FOR VIRTUAL REALITY LIGHT FIELD BASICS JAMES TOMPKIN
VIDEO FOR VIRTUAL REALITY LIGHT FIELD BASICS JAMES TOMPKIN WHAT IS A LIGHT FIELD? Light field seems to have turned into a catch-all term for many advanced camera/display technologies. WHAT IS A LIGHT FIELD?
Photo Tourism: Exploring Photo Collections in 3D
Photo Tourism: Exploring Photo Collections in 3D SIGGRAPH 2006 Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research 2006 2006 Noah Snavely Noah Snavely Reproduced with
A million pixels, a million polygons. Which is heavier? François X. Sillion. imagis* Grenoble, France
A million pixels, a million polygons. Which is heavier? François X. Sillion imagis* Grenoble, France *A joint research project of CNRS, INRIA, INPG and UJF MAGIS Why this question? Evolution of processing
Image or Object? Is this real?
Image or Object? Michael F. Cohen Microsoft Is this real? Photo by Patrick Jennings (patrick@synaptic.bc.ca), Copyright 1995, 96, 97 Whistler B. C. Canada Modeling, Rendering, and Lighting 1 A mental model?
Jingyi Yu CISC 849. Department of Computer and Information Science
Digital Photography and Videos Jingyi Yu CISC 849 Light Fields, Lumigraph, and Image-based Rendering Pinhole Camera A camera captures a set of rays A pinhole camera captures a set of rays passing through
VIDEO-TO-3D. Marc Pollefeys, Luc Van Gool, Maarten Vergauwen, Kurt Cornelis, Frank Verbiest, Jan Tops
VIDEO-TO-3D Marc Pollefeys, Luc Van Gool, Maarten Vergauwen, Kurt Cornelis, Frank Verbiest, Jan Tops Center for Processing of Speech and Images, K.U.Leuven Dept. of Computer Science, University of North
A million pixels, a million polygons: which is heavier?
A million pixels, a million polygons: which is heavier? François X. Sillion To cite this version: François X. Sillion. A million pixels, a million polygons: which is heavier?. Eurographics 97, Sep 1997,
Project Periodic Report Summary
Project Periodic Report Summary Month 12 Date: 24.7.2014 Grant Agreement number: EU 323567 Project acronym: HARVEST4D Project title: Harvesting Dynamic 3D Worlds from Commodity Sensor Clouds TABLE OF CONTENTS
Image Based Lighting with Near Light Sources
Image Based Lighting with Near Light Sources Shiho Furuya, Takayuki Itoh Graduate School of Humanitics and Sciences, Ochanomizu University E-mail: {shiho, itot}@itolab.is.ocha.ac.jp Abstract Recent some
Image Based Lighting with Near Light Sources
Image Based Lighting with Near Light Sources Shiho Furuya, Takayuki Itoh Graduate School of Humanitics and Sciences, Ochanomizu University E-mail: {shiho, itot}@itolab.is.ocha.ac.jp Abstract Recent some
Real-time Generation and Presentation of View-dependent Binocular Stereo Images Using a Sequence of Omnidirectional Images
Real-time Generation and Presentation of View-dependent Binocular Stereo Images Using a Sequence of Omnidirectional Images Abstract This paper presents a new method to generate and present arbitrarily
Image Base Rendering: An Introduction
Image Base Rendering: An Introduction Cliff Lindsay CS563 Spring 03, WPI 1. Introduction Up to this point, we have focused on showing 3D objects in the form of polygons. This is not the only approach to
Free-viewpoint video renderer
jgt08 2008/9/26 14:24 page 1 #1 Vol. [VOL], No. [ISS]: 1 13 Free-viewpoint video renderer J. Starck, J. Kilner, and A. Hilton Centre for Vision, Speech and Signal Processing, University of Surrey, UK.
Image-Based Rendering. Johns Hopkins Department of Computer Science Course : Rendering Techniques, Professor: Jonathan Cohen
Image-Based Rendering Image-Based Rendering What is it? Still a difficult question to answer Uses images (photometric( info) as key component of model representation What s Good about IBR Model acquisition
CSc Topics in Computer Graphics 3D Photography
CSc 83010 Topics in Computer Graphics 3D Photography Tuesdays 11:45-1:45 1:45 Room 3305 Ioannis Stamos istamos@hunter.cuny.edu Office: 1090F, Hunter North (Entrance at 69 th bw/ / Park and Lexington Avenues)
Image-Based Modeling and Rendering
Traditional Computer Graphics Image-Based Modeling and Rendering Thomas Funkhouser Princeton University COS 426 Guest Lecture Spring 2003 How would you model and render this scene? (Jensen) How about this
Depth Synthesis and Local Warps for Plausible Image-based Navigation
Depth Synthesis and Local Warps for Plausible Image-based Navigation GAURAV CHAURASIA and SYLVAIN DUCHENE REVES/INRIA Sophia Antipolis and OLGA SORKINE-HORNUNG ETH Zurich and GEORGE DRETTAKIS REVES/INRIA
Processing 3D Surface Data
Processing 3D Surface Data Computer Animation and Visualisation Lecture 12 Institute for Perception, Action & Behaviour School of Informatics 3D Surfaces 1 3D surface data... where from? Iso-surfacing
Online Interactive 4D Character Animation
Online Interactive 4D Character Animation Marco Volino, Peng Huang and Adrian Hilton Web3D 2015 Outline 4D Performance Capture - 3D Reconstruction, Alignment, Texture Maps Animation - Parametric Motion
A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India
A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India Keshav Mahavidyalaya, University of Delhi, Delhi, India Abstract
What is Computer Vision? Introduction. We all make mistakes. Why is this hard? What was happening. What do you see? Intro Computer Vision
What is Computer Vision? Trucco and Verri (Text): Computing properties of the 3-D world from one or more digital images Introduction Introduction to Computer Vision CSE 152 Lecture 1 Sockman and Shapiro:
Sashi Kumar Penta COMP Final Project Report Department of Computer Science, UNC at Chapel Hill 13 Dec, 2006
Computer vision framework for adding CG Simulations Sashi Kumar Penta sashi@cs.unc.edu COMP 790-072 Final Project Report Department of Computer Science, UNC at Chapel Hill 13 Dec, 2006 Figure 1: (i) Top
Real Time Rendering. CS 563 Advanced Topics in Computer Graphics. Songxiang Gu Jan, 31, 2005
Real Time Rendering CS 563 Advanced Topics in Computer Graphics Songxiang Gu Jan, 31, 2005 Introduction Polygon based rendering Phong modeling Texture mapping Opengl, Directx Point based rendering VTK
Multi-view stereo. Many slides adapted from S. Seitz
Multi-view stereo Many slides adapted from S. Seitz Beyond two-view stereo The third eye can be used for verification Multiple-baseline stereo Pick a reference image, and slide the corresponding window
Image-Based Rendering and Light Fields
CS194-13: Advanced Computer Graphics Lecture #9 Image-Based Rendering University of California Berkeley Image-Based Rendering and Light Fields Lecture #9: Wednesday, September 30th 2009 Lecturer: Ravi
EE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry
3D Models from Range Sensors. Gianpaolo Palma
3D Models from Range Sensors Gianpaolo Palma Who Gianpaolo Palma Researcher at Visual Computing Laboratory (ISTI-CNR) Expertise: 3D scanning, Mesh Processing, Computer Graphics E-mail: gianpaolo.palma@isti.cnr.it
Multi-View Stereo for Static and Dynamic Scenes
Multi-View Stereo for Static and Dynamic Scenes Wolfgang Burgard Jan 6, 2010 Main references Yasutaka Furukawa and Jean Ponce, Accurate, Dense and Robust Multi-View Stereopsis, 2007 C.L. Zitnick, S.B.
Neue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen?
Neue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen? Fraunhofer HHI 13.07.2017 1 Fraunhofer-Gesellschaft Fraunhofer is Europe s largest organization for applied research.
Image-Based Rendering. Johns Hopkins Department of Computer Science Course : Rendering Techniques, Professor: Jonathan Cohen
Image-Based Rendering Image-Based Rendering What is it? Still a difficult question to answer Uses images (photometric( info) as key component of model representation What s Good about IBR Model acquisition
Image-Based Deformation of Objects in Real Scenes
Image-Based Deformation of Objects in Real Scenes Han-Vit Chung and In-Kwon Lee Dept. of Computer Science, Yonsei University sharpguy@cs.yonsei.ac.kr, iklee@yonsei.ac.kr Abstract. We present a new method
Structure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera
DEPTH, STEREO AND FOCUS WITH LIGHT-FIELD CAMERAS
DEPTH, STEREO AND FOCUS WITH LIGHT-FIELD CAMERAS CINEC 2014 Frederik Zilly, Head of Group Computational Imaging & Algorithms Moving Picture Technologies Department, Fraunhofer IIS Fraunhofer, Frederik
Structure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t R 2 3,t 3 Camera 1 Camera
Computational Imaging for Self-Driving Vehicles
CVPR 2018 Computational Imaging for Self-Driving Vehicles Jan Kautz--------Ramesh Raskar--------Achuta Kadambi--------Guy Satat Computational Imaging for Self-Driving Vehicles Jan Kautz--------Ramesh Raskar--------Achuta
Structured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov
Structured Light II Johannes Köhler Johannes.koehler@dfki.de Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov Introduction Previous lecture: Structured Light I Active Scanning Camera/emitter
Hybrid Rendering for Collaborative, Immersive Virtual Environments
Hybrid Rendering for Collaborative, Immersive Virtual Environments Stephan Würmlin wuermlin@inf.ethz.ch Outline! Rendering techniques GBR, IBR and HR! From images to models! Novel view generation! Putting
Cybercity Walker - Layered Morphing Method -
Cybercity Walker - Layered Morphing Method - Takaaki Endo 1), Akihiro Katayama 1), Hideyuki Tamura 1), Michitaka Hirose 2), and Tomohiro Tanikawa 2) 1) Mixed Reality Systems Laboratory Inc. 6-145 Hanasakicho,
Linearizing the Plenoptic Space
Linearizing the Plenoptic Space Grégoire Nieto1, Frédéric Devernay1, James Crowley2 LJK, Université Grenoble Alpes, France 2 LIG, Université Grenoble Alpes, France 1 1 Goal: synthesize a new view Capture/sample
Passive 3D Photography
SIGGRAPH 2000 Course on 3D Photography Passive 3D Photography Steve Seitz Carnegie Mellon University University of Washington http://www.cs cs.cmu.edu/~ /~seitz Visual Cues Shading Merle Norman Cosmetics,
An Algorithm for Seamless Image Stitching and Its Application
An Algorithm for Seamless Image Stitching and Its Application Jing Xing, Zhenjiang Miao, and Jing Chen Institute of Information Science, Beijing JiaoTong University, Beijing 100044, P.R. China Abstract.
Automated Processing for 3D Mosaic Generation, a Change of Paradigm
Automated Processing for 3D Mosaic Generation, a Change of Paradigm Frank BIGNONE, Japan Key Words: 3D Urban Model, Street Imagery, Oblique imagery, Mobile Mapping System, Parallel processing, Digital
Tutorial (Beginner level): Orthomosaic and DEM Generation with Agisoft PhotoScan Pro 1.3 (with Ground Control Points)
Tutorial (Beginner level): Orthomosaic and DEM Generation with Agisoft PhotoScan Pro 1.3 (with Ground Control Points) Overview Agisoft PhotoScan Professional allows to generate georeferenced dense point
3D Object Representations. COS 526, Fall 2016 Princeton University
3D Object Representations COS 526, Fall 2016 Princeton University 3D Object Representations How do we... Represent 3D objects in a computer? Acquire computer representations of 3D objects? Manipulate computer
ENGN D Photography / Spring 2018 / SYLLABUS
ENGN 2502 3D Photography / Spring 2018 / SYLLABUS Description of the proposed course Over the last decade digital photography has entered the mainstream with inexpensive, miniaturized cameras routinely
Depth Images: Representations and Real-time Rendering
Depth Images: Representations and Real-time Rendering Pooja Verlani, Aditi Goswami, P. J. Narayanan Centre for Visual Information Technology IIIT, Hyderabad 2 INDIA pjn@iiit.ac.in Shekhar Dwivedi GE JFWTC
V-Sentinel: A Novel Framework for Situational Awareness and Surveillance
V-Sentinel: A Novel Framework for Situational Awareness and Surveillance Suya You Integrated Media Systems Center Computer Science Department University of Southern California March 2005 1 Objective Developing
IMAGE BASED RENDERING: Using High Dynamic Range Photographs to Light Architectural Scenes
IMAGE BASED RENDERING 1 IMAGE BASED RENDERING: Using High Dynamic Range Photographs to Light Architectural Scenes KATHLEEN CHENEY MEHLIKA INANICI University of Washington Abstract Image Based Rendering
More and More on Light Fields. Last Lecture
More and More on Light Fields Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 4 Last Lecture Re-review with emphasis on radiometry Mosaics & Quicktime VR The Plenoptic function The main
View Dependent Layered Projective Texture Maps
View Dependent Layered Projective Texture Maps Alex Reche, George Drettakis To cite this version: Alex Reche, George Drettakis. View Dependent Layered Projective Texture Maps. J. Rokne and R. Klein and
Synthesizing Realistic Facial Expressions from Photographs
Synthesizing Realistic Facial Expressions from Photographs 1998 F. Pighin, J Hecker, D. Lischinskiy, R. Szeliskiz and D. H. Salesin University of Washington, The Hebrew University Microsoft Research 1
3D Computer Vision. Structured Light II. Prof. Didier Stricker. Kaiserlautern University.
3D Computer Vision Structured Light II Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Introduction
AUTOMATED 3D MODELING OF URBAN ENVIRONMENTS
AUTOMATED 3D MODELING OF URBAN ENVIRONMENTS Ioannis Stamos Department of Computer Science Hunter College, City University of New York 695 Park Avenue, New York NY 10065 istamos@hunter.cuny.edu http://www.cs.hunter.cuny.edu/
Processing 3D Surface Data
Processing 3D Surface Data Computer Animation and Visualisation Lecture 17 Institute for Perception, Action & Behaviour School of Informatics 3D Surfaces 1 3D surface data... where from? Iso-surfacing
Modeling Light. Michal Havlik : Computational Photography Alexei Efros, CMU, Fall 2007
Modeling Light Michal Havlik 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 The Plenoptic Function Figure by Leonard McMillan Q: What is the set of all things that we can ever see? A: The
Shape and Appearance from Images and Range Data
SIGGRAPH 2000 Course on 3D Photography Shape and Appearance from Images and Range Data Brian Curless University of Washington Overview Range images vs. point clouds Registration Reconstruction from point
VOLUMETRIC VIDEO // PLENOPTIC LIGHTFIELD // MULTI CAMERA METHODOLOGIES JORDAN HALSEY // VR PLAYHOUSE
VOLUMETRIC VIDEO // PLENOPTIC LIGHTFIELD // MULTI CAMERA METHODOLOGIES JORDAN HALSEY // VR PLAYHOUSE VOLUMETRIC VIDEO // PLENOPTIC LIGHTFIELD // MULTI CAMERA METHODOLOGIES Pro: Highly realistic seated
Fast and robust techniques for 3D/2D registration and photo blending on massive point clouds
www.crs4.it/vic/ vcg.isti.cnr.it/ Fast and robust techniques for 3D/2D registration and photo blending on massive point clouds R. Pintus, E. Gobbetti, M.Agus, R. Combet CRS4 Visual Computing M. Callieri
Overview. Augmented reality and applications Marker-based augmented reality. Camera model. Binary markers Textured planar markers
Augmented reality Overview Augmented reality and applications Marker-based augmented reality Binary markers Textured planar markers Camera model Homography Direct Linear Transformation What is augmented
Volumetric Scene Reconstruction from Multiple Views
Volumetric Scene Reconstruction from Multiple Views Chuck Dyer University of Wisconsin dyer@cs cs.wisc.edu www.cs cs.wisc.edu/~dyer Image-Based Scene Reconstruction Goal Automatic construction of photo-realistic
Surface reconstruction Introduction. Florent Lafarge Inria Sophia Antipolis - Mediterranee
Surface reconstruction Introduction Florent Lafarge Inria Sophia Antipolis - Mediterranee Outline Contents Introduction Smooth/piecewise-smooth reconstruction methods Primitive-based reconstruction methods
3D Modeling of Objects Using Laser Scanning
1 3D Modeling of Objects Using Laser Scanning D. Jaya Deepu, LPU University, Punjab, India Email: Jaideepudadi@gmail.com Abstract: In the last few decades, constructing accurate three-dimensional models
High-Fidelity Augmented Reality Interactions Hrvoje Benko Researcher, MSR Redmond
High-Fidelity Augmented Reality Interactions Hrvoje Benko Researcher, MSR Redmond New generation of interfaces Instead of interacting through indirect input devices (mice and keyboard), the user is interacting
Interactive Vizualisation of Complex Real-World Light Sources
Interactive Vizualisation of Complex Real-World Light Sources Xavier Granier(1-2) Michael Goesele(3) (2) (3) Wolgang Heidrich Hans-Peter Seidel (1) (2) (3) IPARLA/INRIA LaBRI (Bordeaux) University of British
Some Thoughts on Visibility
Some Thoughts on Visibility Frédo Durand MIT Lab for Computer Science Visibility is hot! 4 papers at Siggraph 4 papers at the EG rendering workshop A wonderful dedicated workshop in Corsica! A big industrial
DECODING COMPLEXITY CONSTRAINED RATE-DISTORTION OPTIMIZATION FOR THE COMPRESSION OF CONCENTRIC MOSAICS
DECODING COMPLEXITY CONSTRAINED RATE-DISTORTION OPTIMIZATION FOR THE COMPRESSION OF CONCENTRIC MOSAICS Eswar Kalyan Vutukuri, Ingo Bauerman and Eckehard Steinbach Institute of Communication Networks, Media
BIL Computer Vision Apr 16, 2014
BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm
Ping Tan. Simon Fraser University
Ping Tan Simon Fraser University Photos vs. Videos (live photos) A good photo tells a story Stories are better told in videos Videos in the Mobile Era (mobile & share) More videos are captured by mobile
Il colore: acquisizione e visualizzazione. Lezione 17: 11 Maggio 2012
Il colore: acquisizione e visualizzazione Lezione 17: 11 Maggio 2012 The importance of color information Precision vs. Perception 3D scanned geometry Photo Color and appearance Pure geometry Pure color
IMMERSIVE VIRTUAL REALITY FOR LIVE-ACTION VIDEO USING CAMERA ARRAYS
IMMERSIVE VIRTUAL REALITY FOR LIVE-ACTION VIDEO USING CAMERA ARRAYS Matthias Ziegler, Joachim Keinert, Nina Holzer, Thorsten Wolf, Tobias Jaschke, Ron op het Veld, Faezeh Sadat Zakeri and Siegfried Foessel
Towards a Perceptual Method of Blending for Image-Based Models
Towards a Perceptual Method of Blending for Image-Based Models Gordon Watson, Patrick O Brien and Mark Wright Edinburgh Virtual Environment Centre University of Edinburgh JCMB, Mayfield Road, Edinburgh
Faces and Image-Based Lighting
Announcements Faces and Image-Based Lighting Project #3 artifacts voting Final project: Demo on 6/25 (Wednesday) 13:30pm in this room Reports and videos due on 6/26 (Thursday) 11:59pm Digital Visual Effects,
Image-Based Lighting. Inserting Synthetic Objects
Image-Based Lighting 15-463: Rendering and Image Processing Alexei Efros with a lot of slides donated by Paul Debevec Inserting Synthetic Objects Why does this look so bad? Wrong camera orientation Wrong
Tutorial (Beginner level): Orthomosaic and DEM Generation with Agisoft PhotoScan Pro 1.3 (without Ground Control Points)
Tutorial (Beginner level): Orthomosaic and DEM Generation with Agisoft PhotoScan Pro 1.3 (without Ground Control Points) Overview Agisoft PhotoScan Professional allows to generate georeferenced dense point
Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza
Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time, ICRA 14, by Pizzoli, Forster, Scaramuzza [M. Pizzoli, C. Forster,
Ray tracing based fast refraction method for an object seen through a cylindrical glass
20th International Congress on Modelling and Simulation, Adelaide, Australia, 1 6 December 2013 www.mssanz.org.au/modsim2013 Ray tracing based fast refraction method for an object seen through a cylindrical
Additional Material (electronic only)
Additional Material (electronic only) This additional material contains a presentation of additional capabilities of the system, a discussion of performance and temporal coherence as well as other limitations.
Image-Based Rendering. Image-Based Rendering
Image-Based Rendering Image-Based Rendering What is it? Still a difficult question to answer Uses images (photometric info) as key component of model representation 1 What s Good about IBR Model acquisition
Structure from Motion and Multi- view Geometry. Last lecture
Structure from Motion and Multi- view Geometry Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 5 Last lecture S. J. Gortler, R. Grzeszczuk, R. Szeliski,M. F. Cohen The Lumigraph, SIGGRAPH,
Collaborative Mapping with Streetlevel Images in the Wild. Yubin Kuang Co-founder and Computer Vision Lead
Collaborative Mapping with Streetlevel Images in the Wild Yubin Kuang Co-founder and Computer Vision Lead Mapillary Mapillary is a street-level imagery platform, powered by collaboration and computer vision.
Modeling and Analyzing 3D Shapes using Clues from 2D Images. Minglun Gong Dept. of CS, Memorial Univ.
Modeling and Analyzing 3D Shapes using Clues from 2D Images Minglun Gong Dept. of CS, Memorial Univ. Modeling Flowers from Images Reconstructing 3D flower petal shapes from a single image is difficult
Multi-View 3D-Reconstruction
Multi-View 3D-Reconstruction Cedric Cagniart Computer Aided Medical Procedures (CAMP) Technische Universität München, Germany 1 Problem Statement Given several calibrated views of an object... can we automatically
Multiview Depth-Image Compression Using an Extended H.264 Encoder Morvan, Y.; Farin, D.S.; de With, P.H.N.
Multiview Depth-Image Compression Using an Extended H.264 Encoder Morvan, Y.; Farin, D.S.; de With, P.H.N. Published in: Proceedings of the 9th international conference on Advanced Concepts for Intelligent
5LSH0 Advanced Topics Video & Analysis
1 Multiview 3D video / Outline 2 Advanced Topics Multimedia Video (5LSH0), Module 02 3D Geometry, 3D Multiview Video Coding & Rendering Peter H.N. de With, Sveta Zinger & Y. Morvan ( p.h.n.de.with@tue.nl
TREE-STRUCTURED ALGORITHM FOR EFFICIENT SHEARLET-DOMAIN LIGHT FIELD RECONSTRUCTION. Suren Vagharshakyan, Robert Bregovic, Atanas Gotchev
TREE-STRUCTURED ALGORITHM FOR EFFICIENT SHEARLET-DOMAIN LIGHT FIELD RECONSTRUCTION Suren Vagharshakyan, Robert Bregovic, Atanas Gotchev Department of Signal Processing, Tampere University of Technology,
CS 684 Fall 2005 Image-based Modeling and Rendering. Ruigang Yang
CS 684 Fall 2005 Image-based Modeling and Rendering Ruigang Yang Administrivia Classes: Monday and Wednesday, 4:00-5:15 PM Instructor: Ruigang Yang ryang@cs.uky.edu Office Hour: Robotics 514D, MW 1500-1600
Large Scale 3D Reconstruction by Structure from Motion
Large Scale 3D Reconstruction by Structure from Motion Devin Guillory Ziang Xie CS 331B 7 October 2013 Overview Rome wasn t built in a day Overview of SfM Building Rome in a Day Building Rome on a Cloudless
Image-Based Rendering using Image-Warping Motivation and Background
Image-Based Rendering using Image-Warping Motivation and Background Leonard McMillan LCS Computer Graphics Group MIT The field of three-dimensional computer graphics has long focused on the problem of
3D from Images - Assisted Modeling, Photogrammetry. Marco Callieri ISTI-CNR, Pisa, Italy
3D from Images - Assisted Modeling, Photogrammetry Marco Callieri ISTI-CNR, Pisa, Italy 3D from Photos Our not-so-secret dream: obtain a reliable and precise 3D from simple photos Why? Easier, cheaper
Technical Papers November 2017 Bangkok Int'l Trade & Exhibition Centre (BITEC)
Session: Form, Furniture and Fashion Date/Time: 28 November 2017, 09:00am - 10:45am Position-Based Tensegrity Design Material-Minimizing Forms and Structures Reconfigurable Interlocking Furniture Fabricable