3D Reconstruction with Tango. Ivan Dryanovski, Google Inc.

Size: px
Start display at page:

Download "3D Reconstruction with Tango. Ivan Dryanovski, Google Inc."

Transcription

1 3D Reconstruction with Tango Ivan Dryanovski, Google Inc.

2 Contents Problem statement and motivation The Tango SDK 3D reconstruction - data structures & algorithms Applications Developer tools

3 Problem formulation Goal: Create accurate, textured 3D models of indoor spaces real-time, on mobile device offline, in the cloud Motivation immersive VR / AR - gaming, professional apps large scale maps, floorpans - navigation

4 Problem formulation Challenges: Partial, noisy depth data Memory and computational constraints Platform diversity

5 Tango software stack Motion Tracking Area Learning Depth perception

6 Klingensmith, Matthew, et al. "Chisel: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device using Spatially Hashed Signed Distance Fields." Robotics: Science and Systems XI (2015). Tango 3D reconstruction Real-time, online process 6DoF Pose 100Hz Volumetric fusion Mesh extraction Application tasks Color images Depth images 10Hz 10Hz using two-tier TSDF voxel grid 10Hz using Marching Cubes 10Hz rendering occlusions path planning collision detection

7 Tango 3D reconstruction Offline process 6DoF 6DoF Pose Pose (IMU (IMU 6DoF + features) features) Pose Volumetric fusion Mesh extraction Mesh simplification Mesh texturing Application tasks Color Color Color images Depth images using two-tier TSDF voxel grid using Marching Cubes vertex decimation texture atlas generation projective image texturing rendering occlusions path planning collision detection

8 Curless, Brian, and Marc Levoy. "A volumetric method for building complex models from range images." Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, Signed Distance Function (SDF) Represent space using voxel grid Voxel contains signed distance function to nearest surface (SDF) Only update near surface - estimate truncated function (TSDF) Grid is updated by fusing depth readings (using running weighted average filter)

9 Curless, Brian, and Marc Levoy. "A volumetric method for building complex models from range images." Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, Signed Distance Function (SDF) Surface of objects is represented by the zero-isosurface in voxel grid Traversing the isosurface is done using Marching Cubes Output is a triangle mesh

10 Voxel memory layout 64-bit per voxel (includes TSDF value, color value, filter weights)

11 Nießner, Matthias, et al. "Real-time 3D reconstruction at scale using voxel hashing." ACM Transactions on Graphics (TOG) 32.6 (2013): 169. Dynamic Two-Tier TSDF voxel grid Single TSDF grid for the entire scene would require too much memory Hierarchical grid - larger volumes contain sub-grids Volumes are sparsely allocated, and indexed by a hash map

12 Klingensmith, Matthew, et al. "Chisel: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device using Spatially Hashed Signed Distance Fields." Robotics: Science and Systems XI (2015). Local frustum culling When adding new observation, do intersect the depth camera frustum with the coarse volume grid New volumes are allocated ondemand

13 Klingensmith, Matthew, et al. "Chisel: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device using Spatially Hashed Signed Distance Fields." Robotics: Science and Systems XI (2015). Local frustum culling Only intersected volumes receive TSDF updates Only intersected volumes have their mesh segments re-extracted Allows building of large models while updating and streaming only local segments

14

15

16

17

18 Large-scale reconstruction 25 individual trajectories (~3 hrs) Co-located using area learning Final 3D map (false color by height)

19 Large-scale reconstruction Ivan sits here

20 Developer tools Tango Client API C / Java / Unity 6DoF pose, images, depth

21 Developer tools Tango Client API C / Java / Unity 6DoF pose, images, depth Tango 3D Reconstruction API C / Java / Unity provides textured 3D meshes

22 Developer tools Constructor Android app for building and sharing 3D models Tango Client API C / Java / Unity 6DoF pose, images, depth Tango 3D Reconstruction API C / Java / Unity provides textured 3D meshes

23 GPU implementation If no shared memory between CPU & GPU, copying data is a bottleneck 6DoF Pose 100Hz Volumetric fusion Mesh extraction Application tasks Color images Depth images 10Hz 10Hz using two-tier TSDF voxel grid 10Hz using Marching Cubes 10Hz COPY rendering occlusions path planning collision detection COPY Depth images as graphics texture? Collision calculations on graphics hardware?

24 Thank you! Q&A Ivan Dryanovski, Google Inc.

Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting

Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting R. Maier 1,2, K. Kim 1, D. Cremers 2, J. Kautz 1, M. Nießner 2,3 Fusion Ours 1

More information

Large-scale, real-time 3D scene reconstruction on a mobile device

Large-scale, real-time 3D scene reconstruction on a mobile device Auton Robot (2017) 41:1423 1445 DOI 10.1007/s10514-017-9624-2 Large-scale, real-time 3D scene reconstruction on a mobile device Ivan Dryanovski 1 Matthew Klingensmith 2 Siddhartha S. Srinivasa 2 Jizhong

More information

Outline. 1 Why we re interested in Real-Time tracking and mapping. 3 Kinect Fusion System Overview. 4 Real-time Surface Mapping

Outline. 1 Why we re interested in Real-Time tracking and mapping. 3 Kinect Fusion System Overview. 4 Real-time Surface Mapping Outline CSE 576 KinectFusion: Real-Time Dense Surface Mapping and Tracking PhD. work from Imperial College, London Microsoft Research, Cambridge May 6, 2013 1 Why we re interested in Real-Time tracking

More information

MultiAR Project Michael Pekel, Ofir Elmakias [GIP] [234329]

MultiAR Project Michael Pekel, Ofir Elmakias [GIP] [234329] MultiAR Project Michael Pekel, Ofir Elmakias [GIP] [234329] Supervisors Dr. Matan Sela Mr. Yaron Honen Assistants Alexander Porotskiy Summary MultiAR is a multiplayer quest (Outdoor Real Time Multiplayer

More information

Hierarchical Volumetric Fusion of Depth Images

Hierarchical Volumetric Fusion of Depth Images Hierarchical Volumetric Fusion of Depth Images László Szirmay-Kalos, Milán Magdics Balázs Tóth, Tamás Umenhoffer Real-time color & 3D information Affordable integrated depth and color cameras Application:

More information

Many rendering scenarios, such as battle scenes or urban environments, require rendering of large numbers of autonomous characters.

Many rendering scenarios, such as battle scenes or urban environments, require rendering of large numbers of autonomous characters. 1 2 Many rendering scenarios, such as battle scenes or urban environments, require rendering of large numbers of autonomous characters. Crowd rendering in large environments presents a number of challenges,

More information

ABSTRACT. KinectFusion is a surface reconstruction method to allow a user to rapidly

ABSTRACT. KinectFusion is a surface reconstruction method to allow a user to rapidly ABSTRACT Title of Thesis: A REAL TIME IMPLEMENTATION OF 3D SYMMETRIC OBJECT RECONSTRUCTION Liangchen Xi, Master of Science, 2017 Thesis Directed By: Professor Yiannis Aloimonos Department of Computer Science

More information

Introduction to Mobile Robotics Techniques for 3D Mapping

Introduction to Mobile Robotics Techniques for 3D Mapping Introduction to Mobile Robotics Techniques for 3D Mapping Wolfram Burgard, Michael Ruhnke, Bastian Steder 1 Why 3D Representations Robots live in the 3D world. 2D maps have been applied successfully for

More information

Project Updates Short lecture Volumetric Modeling +2 papers

Project Updates Short lecture Volumetric Modeling +2 papers Volumetric Modeling Schedule (tentative) Feb 20 Feb 27 Mar 5 Introduction Lecture: Geometry, Camera Model, Calibration Lecture: Features, Tracking/Matching Mar 12 Mar 19 Mar 26 Apr 2 Apr 9 Apr 16 Apr 23

More information

Monocular Tracking and Reconstruction in Non-Rigid Environments

Monocular Tracking and Reconstruction in Non-Rigid Environments Monocular Tracking and Reconstruction in Non-Rigid Environments Kick-Off Presentation, M.Sc. Thesis Supervisors: Federico Tombari, Ph.D; Benjamin Busam, M.Sc. Patrick Ruhkamp 13.01.2017 Introduction Motivation:

More information

Surface Reconstruction. Gianpaolo Palma

Surface Reconstruction. Gianpaolo Palma Surface Reconstruction Gianpaolo Palma Surface reconstruction Input Point cloud With or without normals Examples: multi-view stereo, union of range scan vertices Range scans Each scan is a triangular mesh

More information

A Volumetric Method for Building Complex Models from Range Images

A Volumetric Method for Building Complex Models from Range Images A Volumetric Method for Building Complex Models from Range Images Brian Curless Marc Levoy Computer Graphics Laboratory Stanford University Introduction Goal Given a set of aligned, dense range images,

More information

Complex Models from Range Images. A Volumetric Method for Building. Brian Curless. Marc Levoy. Computer Graphics Laboratory. Stanford University

Complex Models from Range Images. A Volumetric Method for Building. Brian Curless. Marc Levoy. Computer Graphics Laboratory. Stanford University A Volumetric Method for Building Complex Models from Range Images Computer Graphics Laboratory Stanford University Brian Curless Marc Levoy Introduction Goal Given a set of aligned, dense range images,

More information

Geometric Reconstruction Dense reconstruction of scene geometry

Geometric Reconstruction Dense reconstruction of scene geometry Lecture 5. Dense Reconstruction and Tracking with Real-Time Applications Part 2: Geometric Reconstruction Dr Richard Newcombe and Dr Steven Lovegrove Slide content developed from: [Newcombe, Dense Visual

More information

Scanning and Printing Objects in 3D

Scanning and Printing Objects in 3D Scanning and Printing Objects in 3D Dr. Jürgen Sturm metaio GmbH (formerly Technical University of Munich) My Research Areas Visual navigation for mobile robots RoboCup Kinematic Learning Articulated Objects

More information

Scanning and Printing Objects in 3D Jürgen Sturm

Scanning and Printing Objects in 3D Jürgen Sturm Scanning and Printing Objects in 3D Jürgen Sturm Metaio (formerly Technical University of Munich) My Research Areas Visual navigation for mobile robots RoboCup Kinematic Learning Articulated Objects Quadrocopters

More information

1 wajdibr.com/ @WajdiBenRabah #MIXIT18 Augmented Reality A way of fusing the real and the virtual world by overlaying digital data on to real-world analogue views. Anyone with a mobile phone or laptop

More information

3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller

3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller 3D Computer Vision Depth Cameras Prof. Didier Stricker Oliver Wasenmüller Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de

More information

Hybrid Rendering for Collaborative, Immersive Virtual Environments

Hybrid Rendering for Collaborative, Immersive Virtual Environments Hybrid Rendering for Collaborative, Immersive Virtual Environments Stephan Würmlin wuermlin@inf.ethz.ch Outline! Rendering techniques GBR, IBR and HR! From images to models! Novel view generation! Putting

More information

Emerging Vision Technologies: Enabling a New Era of Intelligent Devices

Emerging Vision Technologies: Enabling a New Era of Intelligent Devices Emerging Vision Technologies: Enabling a New Era of Intelligent Devices Computer vision overview Computer vision is being integrated in our daily lives Acquiring, processing, and understanding visual data

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem

More information

The Application Stage. The Game Loop, Resource Management and Renderer Design

The Application Stage. The Game Loop, Resource Management and Renderer Design 1 The Application Stage The Game Loop, Resource Management and Renderer Design Application Stage Responsibilities 2 Set up the rendering pipeline Resource Management 3D meshes Textures etc. Prepare data

More information

CS248. Game Mechanics

CS248. Game Mechanics CS248 Game Mechanics INTRODUCTION TOM WANG 2007 BS/MS CS KEY GAME MECHANICS * * * * * WORLD BUILDING CONTROLS CAMERA AI PERFORMANCE WORLD BUILDING WORLD BUILDING Set the atmosphere and tone of the game.

More information

DiFi: Distance Fields - Fast Computation Using Graphics Hardware

DiFi: Distance Fields - Fast Computation Using Graphics Hardware DiFi: Distance Fields - Fast Computation Using Graphics Hardware Avneesh Sud Dinesh Manocha UNC-Chapel Hill http://gamma.cs.unc.edu/difi Distance Fields Distance Function For a site a scalar function f:r

More information

CS 563 Advanced Topics in Computer Graphics QSplat. by Matt Maziarz

CS 563 Advanced Topics in Computer Graphics QSplat. by Matt Maziarz CS 563 Advanced Topics in Computer Graphics QSplat by Matt Maziarz Outline Previous work in area Background Overview In-depth look File structure Performance Future Point Rendering To save on setup and

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction

More information

Geometric and Semantic 3D Reconstruction: Part 4A: Volumetric Semantic 3D Reconstruction. CVPR 2017 Tutorial Christian Häne UC Berkeley

Geometric and Semantic 3D Reconstruction: Part 4A: Volumetric Semantic 3D Reconstruction. CVPR 2017 Tutorial Christian Häne UC Berkeley Geometric and Semantic 3D Reconstruction: Part 4A: Volumetric Semantic 3D Reconstruction CVPR 2017 Tutorial Christian Häne UC Berkeley Dense Multi-View Reconstruction Goal: 3D Model from Images (Depth

More information

Mesh from Depth Images Using GR 2 T

Mesh from Depth Images Using GR 2 T Mesh from Depth Images Using GR 2 T Mairead Grogan & Rozenn Dahyot School of Computer Science and Statistics Trinity College Dublin Dublin, Ireland mgrogan@tcd.ie, Rozenn.Dahyot@tcd.ie www.scss.tcd.ie/

More information

Subdivision Of Triangular Terrain Mesh Breckon, Chenney, Hobbs, Hoppe, Watts

Subdivision Of Triangular Terrain Mesh Breckon, Chenney, Hobbs, Hoppe, Watts Subdivision Of Triangular Terrain Mesh Breckon, Chenney, Hobbs, Hoppe, Watts MSc Computer Games and Entertainment Maths & Graphics II 2013 Lecturer(s): FFL (with Gareth Edwards) Fractal Terrain Based on

More information

3D Object Representations. COS 526, Fall 2016 Princeton University

3D Object Representations. COS 526, Fall 2016 Princeton University 3D Object Representations COS 526, Fall 2016 Princeton University 3D Object Representations How do we... Represent 3D objects in a computer? Acquire computer representations of 3D objects? Manipulate computer

More information

Geometric Modeling. Bing-Yu Chen National Taiwan University The University of Tokyo

Geometric Modeling. Bing-Yu Chen National Taiwan University The University of Tokyo Geometric Modeling Bing-Yu Chen National Taiwan University The University of Tokyo What are 3D Objects? 3D Object Representations What are 3D objects? The Graphics Process 3D Object Representations Raw

More information

Stable Vision-Aided Navigation for Large-Area Augmented Reality

Stable Vision-Aided Navigation for Large-Area Augmented Reality Stable Vision-Aided Navigation for Large-Area Augmented Reality Taragay Oskiper, Han-Pang Chiu, Zhiwei Zhu Supun Samarasekera, Rakesh Teddy Kumar Vision and Robotics Laboratory SRI-International Sarnoff,

More information

Special Topics in Visualization

Special Topics in Visualization Special Topics in Visualization Final Project Report Dual contouring of Hermite Data Submitted By S M Shahed Nejhum 8589-1199 May 19, 2008 Introduction Iso-surface extraction from 3D volumetric data is

More information

W4. Perception & Situation Awareness & Decision making

W4. Perception & Situation Awareness & Decision making W4. Perception & Situation Awareness & Decision making Robot Perception for Dynamic environments: Outline & DP-Grids concept Dynamic Probabilistic Grids Bayesian Occupancy Filter concept Dynamic Probabilistic

More information

LOD and Occlusion Christian Miller CS Fall 2011

LOD and Occlusion Christian Miller CS Fall 2011 LOD and Occlusion Christian Miller CS 354 - Fall 2011 Problem You want to render an enormous island covered in dense vegetation in realtime [Crysis] Scene complexity Many billions of triangles Many gigabytes

More information

Building scalable 3D applications. Ville Miettinen Hybrid Graphics

Building scalable 3D applications. Ville Miettinen Hybrid Graphics Building scalable 3D applications Ville Miettinen Hybrid Graphics What s going to happen... (1/2) Mass market: 3D apps will become a huge success on low-end and mid-tier cell phones Retro-gaming New game

More information

VOLUMETRIC VIDEO // PLENOPTIC LIGHTFIELD // MULTI CAMERA METHODOLOGIES JORDAN HALSEY // VR PLAYHOUSE

VOLUMETRIC VIDEO // PLENOPTIC LIGHTFIELD // MULTI CAMERA METHODOLOGIES JORDAN HALSEY // VR PLAYHOUSE VOLUMETRIC VIDEO // PLENOPTIC LIGHTFIELD // MULTI CAMERA METHODOLOGIES JORDAN HALSEY // VR PLAYHOUSE VOLUMETRIC VIDEO // PLENOPTIC LIGHTFIELD // MULTI CAMERA METHODOLOGIES Pro: Highly realistic seated

More information

Deferred Splatting. Gaël GUENNEBAUD Loïc BARTHE Mathias PAULIN IRIT UPS CNRS TOULOUSE FRANCE.

Deferred Splatting. Gaël GUENNEBAUD Loïc BARTHE Mathias PAULIN IRIT UPS CNRS TOULOUSE FRANCE. Deferred Splatting Gaël GUENNEBAUD Loïc BARTHE Mathias PAULIN IRIT UPS CNRS TOULOUSE FRANCE http://www.irit.fr/~gael.guennebaud Plan Complex Scenes: Triangles or Points? High Quality Splatting: Really

More information

3D Reconstruction from Multi-View Stereo: Implementation Verification via Oculus Virtual Reality

3D Reconstruction from Multi-View Stereo: Implementation Verification via Oculus Virtual Reality 3D Reconstruction from Multi-View Stereo: Implementation Verification via Oculus Virtual Reality Andrew Moran MIT, Class of 2014 andrewmo@mit.edu Ben Eysenbach MIT, Class of 2017 bce@mit.edu Abstract We

More information

Rendering. Converting a 3D scene to a 2D image. Camera. Light. Rendering. View Plane

Rendering. Converting a 3D scene to a 2D image. Camera. Light. Rendering. View Plane Rendering Pipeline Rendering Converting a 3D scene to a 2D image Rendering Light Camera 3D Model View Plane Rendering Converting a 3D scene to a 2D image Basic rendering tasks: Modeling: creating the world

More information

Scene Management. Video Game Technologies 11498: MSc in Computer Science and Engineering 11156: MSc in Game Design and Development

Scene Management. Video Game Technologies 11498: MSc in Computer Science and Engineering 11156: MSc in Game Design and Development Video Game Technologies 11498: MSc in Computer Science and Engineering 11156: MSc in Game Design and Development Chap. 5 Scene Management Overview Scene Management vs Rendering This chapter is about rendering

More information

COMP 4801 Final Year Project. Ray Tracing for Computer Graphics. Final Project Report FYP Runjing Liu. Advised by. Dr. L.Y.

COMP 4801 Final Year Project. Ray Tracing for Computer Graphics. Final Project Report FYP Runjing Liu. Advised by. Dr. L.Y. COMP 4801 Final Year Project Ray Tracing for Computer Graphics Final Project Report FYP 15014 by Runjing Liu Advised by Dr. L.Y. Wei 1 Abstract The goal of this project was to use ray tracing in a rendering

More information

Programmable Shaders for Deformation Rendering

Programmable Shaders for Deformation Rendering Programmable Shaders for Deformation Rendering Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Motivation We present a different way of obtaining mesh deformation. Not a modeling,

More information

Improved 3D Reconstruction using Combined Weighting Strategies

Improved 3D Reconstruction using Combined Weighting Strategies Improved 3D Reconstruction using Combined Weighting Strategies Patrick Stotko Supervised by: Tim Golla Institute of Computer Science II - Computer Graphics University of Bonn Bonn / Germany Abstract Due

More information

Hardware-driven Visibility Culling Jeong Hyun Kim

Hardware-driven Visibility Culling Jeong Hyun Kim Hardware-driven Visibility Culling Jeong Hyun Kim KAIST (Korea Advanced Institute of Science and Technology) Contents Introduction Background Clipping Culling Z-max (Z-min) Filter Programmable culling

More information

Volumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material

Volumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material Volumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material Charles R. Qi Hao Su Matthias Nießner Angela Dai Mengyuan Yan Leonidas J. Guibas Stanford University 1. Details

More information

3D Collision Avoidance for Navigation in Unstructured Environments

3D Collision Avoidance for Navigation in Unstructured Environments 3D Collision Avoidance for Navigation in Unstructured Environments Armin Hornung Humanoid Robots Lab, University of Freiburg May 5, 2011 Motivation Personal robots are to operate in arbitrary complex environments:

More information

Ping Tan. Simon Fraser University

Ping Tan. Simon Fraser University Ping Tan Simon Fraser University Photos vs. Videos (live photos) A good photo tells a story Stories are better told in videos Videos in the Mobile Era (mobile & share) More videos are captured by mobile

More information

Signed Distance Function Representation, Tracking, and Mapping. Tanner Schmidt

Signed Distance Function Representation, Tracking, and Mapping. Tanner Schmidt Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations SDF fusion SDF tracking Related research - KinectFusion Patch Volumes

More information

Point Cloud Filtering using Ray Casting by Eric Jensen 2012 The Basic Methodology

Point Cloud Filtering using Ray Casting by Eric Jensen 2012 The Basic Methodology Point Cloud Filtering using Ray Casting by Eric Jensen 01 The Basic Methodology Ray tracing in standard graphics study is a method of following the path of a photon from the light source to the camera,

More information

Real-Time Volumetric Smoke using D3D10. Sarah Tariq and Ignacio Llamas NVIDIA Developer Technology

Real-Time Volumetric Smoke using D3D10. Sarah Tariq and Ignacio Llamas NVIDIA Developer Technology Real-Time Volumetric Smoke using D3D10 Sarah Tariq and Ignacio Llamas NVIDIA Developer Technology Smoke in NVIDIA s DirectX10 SDK Sample Smoke in the game Hellgate London Talk outline: Why 3D fluid simulation

More information

CS4620/5620: Lecture 14 Pipeline

CS4620/5620: Lecture 14 Pipeline CS4620/5620: Lecture 14 Pipeline 1 Rasterizing triangles Summary 1! evaluation of linear functions on pixel grid 2! functions defined by parameter values at vertices 3! using extra parameters to determine

More information

Wed, October 12, 2011

Wed, October 12, 2011 Practical Occlusion Culling in Killzone 3 Michal Valient Lead Tech, Guerrilla B.V. Talk takeaway Occlusion culling system used in Killzone 3 The reasons why to use software rasterization (Some) technical

More information

Optimizing and Profiling Unity Games for Mobile Platforms. Angelo Theodorou Senior Software Engineer, MPG Gamelab 2014, 25 th -27 th June

Optimizing and Profiling Unity Games for Mobile Platforms. Angelo Theodorou Senior Software Engineer, MPG Gamelab 2014, 25 th -27 th June Optimizing and Profiling Unity Games for Mobile Platforms Angelo Theodorou Senior Software Engineer, MPG Gamelab 2014, 25 th -27 th June 1 Agenda Introduction ARM and the presenter Preliminary knowledge

More information

Modeling, Combining, and Rendering Dynamic Real-World Events From Image Sequences

Modeling, Combining, and Rendering Dynamic Real-World Events From Image Sequences Modeling, Combining, and Rendering Dynamic Real-World Events From Image s Sundar Vedula, Peter Rander, Hideo Saito, and Takeo Kanade The Robotics Institute Carnegie Mellon University Abstract Virtualized

More information

Next-Generation Graphics on Larrabee. Tim Foley Intel Corp

Next-Generation Graphics on Larrabee. Tim Foley Intel Corp Next-Generation Graphics on Larrabee Tim Foley Intel Corp Motivation The killer app for GPGPU is graphics We ve seen Abstract models for parallel programming How those models map efficiently to Larrabee

More information

Massive Model Visualization using Real-time Ray Tracing

Massive Model Visualization using Real-time Ray Tracing Massive Model Visualization using Real-time Ray Tracing Eurographics 2006 Tutorial: Real-time Interactive Massive Model Visualization Andreas Dietrich Philipp Slusallek Saarland University & intrace GmbH

More information

Interactive Computer Graphics A TOP-DOWN APPROACH WITH SHADER-BASED OPENGL

Interactive Computer Graphics A TOP-DOWN APPROACH WITH SHADER-BASED OPENGL International Edition Interactive Computer Graphics A TOP-DOWN APPROACH WITH SHADER-BASED OPENGL Sixth Edition Edward Angel Dave Shreiner Interactive Computer Graphics: A Top-Down Approach with Shader-Based

More information

Speeding up your game

Speeding up your game Speeding up your game The scene graph Culling techniques Level-of-detail rendering (LODs) Collision detection Resources and pointers (adapted by Marc Levoy from a lecture by Tomas Möller, using material

More information

Inside VR on Mobile. Sam Martin Graphics Architect GDC 2016

Inside VR on Mobile. Sam Martin Graphics Architect GDC 2016 Inside VR on Mobile Sam Martin Graphics Architect GDC 2016 VR Today Emerging technology Main mobile VR ecosystems Google Cardboard Samsung GearVR In this talk: Latency Multiple views Performance tuning

More information

Computer Graphics Ray Casting. Matthias Teschner

Computer Graphics Ray Casting. Matthias Teschner Computer Graphics Ray Casting Matthias Teschner Outline Context Implicit surfaces Parametric surfaces Combined objects Triangles Axis-aligned boxes Iso-surfaces in grids Summary University of Freiburg

More information

PERFORMANCE CAPTURE FROM SPARSE MULTI-VIEW VIDEO

PERFORMANCE CAPTURE FROM SPARSE MULTI-VIEW VIDEO Stefan Krauß, Juliane Hüttl SE, SoSe 2011, HU-Berlin PERFORMANCE CAPTURE FROM SPARSE MULTI-VIEW VIDEO 1 Uses of Motion/Performance Capture movies games, virtual environments biomechanics, sports science,

More information

ICP and 3D-Reconstruction

ICP and 3D-Reconstruction N. Slottke, H. Linne 1 Nikolas Slottke 1 Hendrik Linne 2 {7slottke, 7linne}@informatik.uni-hamburg.de Fakultät für Mathematik, Informatik und Naturwissenschaften Technische Aspekte Multimodaler Systeme

More information

Real - Time Rendering. Pipeline optimization. Michal Červeňanský Juraj Starinský

Real - Time Rendering. Pipeline optimization. Michal Červeňanský Juraj Starinský Real - Time Rendering Pipeline optimization Michal Červeňanský Juraj Starinský Motivation Resolution 1600x1200, at 60 fps Hw power not enough Acceleration is still necessary 3.3.2010 2 Overview Application

More information

Clipping. CSC 7443: Scientific Information Visualization

Clipping. CSC 7443: Scientific Information Visualization Clipping Clipping to See Inside Obscuring critical information contained in a volume data Contour displays show only exterior visible surfaces Isosurfaces can hide other isosurfaces Other displays can

More information

IMAGE-BASED RENDERING

IMAGE-BASED RENDERING IMAGE-BASED RENDERING 1. What is Image-Based Rendering? - The synthesis of new views of a scene from pre-recorded pictures.!"$#% "'&( )*+,-/.). #0 1 ' 2"&43+5+, 2. Why? (1) We really enjoy visual magic!

More information

Lecture 10 Dense 3D Reconstruction

Lecture 10 Dense 3D Reconstruction Institute of Informatics Institute of Neuroinformatics Lecture 10 Dense 3D Reconstruction Davide Scaramuzza 1 REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time M. Pizzoli, C. Forster,

More information

Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction

Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction MAIER ET AL.: EFFICIENT LARGE-SCALE ONLINE SURFACE CORRECTION 1 Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction Robert Maier maierr@in.tum.de Raphael Schaller schaller@in.tum.de

More information

Copyright Khronos Group Page 1

Copyright Khronos Group Page 1 Gaming Market Briefing Overview of APIs GDC March 2016 Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem ntrevett@nvidia.com @neilt3d Copyright Khronos Group 2016 - Page 1 Copyright

More information

Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures

Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures Robert Maier, Jörg Stückler, Daniel Cremers International Conference on 3D Vision (3DV) October 2015, Lyon, France Motivation

More information

Minecraft Due: March. 6, 2018

Minecraft Due: March. 6, 2018 CS1950U Topics in 3D Game Engine Development Barbara Meier Minecraft Due: March. 6, 2018 Introduction In this assignment you will build your own version of one of the most popular indie games ever: Minecraft.

More information

ArcGIS Runtime: Building 3D Apps. Rex Hansen Adrien Meriaux

ArcGIS Runtime: Building 3D Apps. Rex Hansen Adrien Meriaux ArcGIS Runtime: Building 3D Apps Rex Hansen Adrien Meriaux Agenda 3D across the ArcGIS Platform 3D in ArcGIS Runtime Road ahead ArcGIS 3D helps customers Create and Manage Design and Simulate Visualize

More information

CVPR 2014 Visual SLAM Tutorial Kintinuous

CVPR 2014 Visual SLAM Tutorial Kintinuous CVPR 2014 Visual SLAM Tutorial Kintinuous kaess@cmu.edu The Robotics Institute Carnegie Mellon University Recap: KinectFusion [Newcombe et al., ISMAR 2011] RGB-D camera GPU 3D/color model RGB TSDF (volumetric

More information

Rendering Grass with Instancing in DirectX* 10

Rendering Grass with Instancing in DirectX* 10 Rendering Grass with Instancing in DirectX* 10 By Anu Kalra Because of the geometric complexity, rendering realistic grass in real-time is difficult, especially on consumer graphics hardware. This article

More information

3D Object Representation. Michael Kazhdan ( /657)

3D Object Representation. Michael Kazhdan ( /657) 3D Object Representation Michael Kazhdan (601.457/657) 3D Objects How can this object be represented in a computer? 3D Objects This one? H&B Figure 10.46 3D Objects This one? H&B Figure 9.9 3D Objects

More information

CSE 167: Introduction to Computer Graphics Lecture #11: Visibility Culling

CSE 167: Introduction to Computer Graphics Lecture #11: Visibility Culling CSE 167: Introduction to Computer Graphics Lecture #11: Visibility Culling Jürgen P. Schulze, Ph.D. University of California, San Diego Fall Quarter 2017 Announcements Project 3 due Monday Nov 13 th at

More information

Volumetric Scene Reconstruction from Multiple Views

Volumetric Scene Reconstruction from Multiple Views Volumetric Scene Reconstruction from Multiple Views Chuck Dyer University of Wisconsin dyer@cs cs.wisc.edu www.cs cs.wisc.edu/~dyer Image-Based Scene Reconstruction Goal Automatic construction of photo-realistic

More information

CS535 Fall Department of Computer Science Purdue University

CS535 Fall Department of Computer Science Purdue University Culling CS535 Fall 2010 Daniel G Aliaga Daniel G. Aliaga Department of Computer Science Purdue University Types of culling View frustum culling Visibility culling e.g., backface culling, portal tlculling,

More information

Generating 3D Colored Face Model Using a Kinect Camera

Generating 3D Colored Face Model Using a Kinect Camera Generating 3D Colored Face Model Using a Kinect Camera Submitted by: Ori Ziskind, Rotem Mordoch, Nadine Toledano Advisors: Matan Sela, Yaron Honen Geometric Image Processing Laboratory, CS, Technion March,

More information

Multiview Reconstruction

Multiview Reconstruction Multiview Reconstruction Why More Than 2 Views? Baseline Too short low accuracy Too long matching becomes hard Why More Than 2 Views? Ambiguity with 2 views Camera 1 Camera 2 Camera 3 Trinocular Stereo

More information

CS 498 VR. Lecture 18-4/4/18. go.illinois.edu/vrlect18

CS 498 VR. Lecture 18-4/4/18. go.illinois.edu/vrlect18 CS 498 VR Lecture 18-4/4/18 go.illinois.edu/vrlect18 Review and Supplement for last lecture 1. What is aliasing? What is Screen Door Effect? 2. How image-order rendering works? 3. If there are several

More information

Image-Based Rendering

Image-Based Rendering Image-Based Rendering COS 526, Fall 2016 Thomas Funkhouser Acknowledgments: Dan Aliaga, Marc Levoy, Szymon Rusinkiewicz What is Image-Based Rendering? Definition 1: the use of photographic imagery to overcome

More information

CS451Real-time Rendering Pipeline

CS451Real-time Rendering Pipeline 1 CS451Real-time Rendering Pipeline JYH-MING LIEN DEPARTMENT OF COMPUTER SCIENCE GEORGE MASON UNIVERSITY Based on Tomas Akenine-Möller s lecture note You say that you render a 3D 2 scene, but what does

More information

Memory-Efficient Interactive Online Reconstruction from Depth Image Streams

Memory-Efficient Interactive Online Reconstruction from Depth Image Streams COPUTER GRAPHICS forum Volume 34 (25), Number 2 pp. 3 emory-efficient Interactive Online Reconstruction from Depth Image Streams F. Reichl, J. Weiss, and R. Westermann Technische Universität ünchen, Computer

More information

Octree-based view-dependent triangle meshes

Octree-based view-dependent triangle meshes Octree-based view-dependent triangle meshes Marta Fairén Department of Software Universitat Politècnica de Catalunya mfairen@lsi.upc.edu Ramón Trueba Department of Software Universitat Politècnica de Catalunya

More information

CSL 859: Advanced Computer Graphics. Dept of Computer Sc. & Engg. IIT Delhi

CSL 859: Advanced Computer Graphics. Dept of Computer Sc. & Engg. IIT Delhi CSL 859: Advanced Computer Graphics Dept of Computer Sc. & Engg. IIT Delhi Point Based Representation Point sampling of Surface Mesh construction, or Mesh-less Often come from laser scanning Or even natural

More information

Minecraft Due: Mar. 1, 2015

Minecraft Due: Mar. 1, 2015 CS1972 Topics in 3D Game Engine Development Barbara Meier Minecraft Due: Mar. 1, 2015 Introduction In this assignment you will build your own version of one of the most popular indie games ever: Minecraft.

More information

Duksu Kim. Professional Experience Senior researcher, KISTI High performance visualization

Duksu Kim. Professional Experience Senior researcher, KISTI High performance visualization Duksu Kim Assistant professor, KORATEHC Education Ph.D. Computer Science, KAIST Parallel Proximity Computation on Heterogeneous Computing Systems for Graphics Applications Professional Experience Senior

More information

Exploiting Depth Camera for 3D Spatial Relationship Interpretation

Exploiting Depth Camera for 3D Spatial Relationship Interpretation Exploiting Depth Camera for 3D Spatial Relationship Interpretation Jun Ye Kien A. Hua Data Systems Group, University of Central Florida Mar 1, 2013 Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships

More information

Positional tracking for VR/AR using Stereo Vision. Edwin AZZAM, CTO Stereolabs

Positional tracking for VR/AR using Stereo Vision. Edwin AZZAM, CTO Stereolabs Positional tracking for VR/AR using Stereo Vision Edwin AZZAM, CTO Stereolabs Outside-in Positional Tracking Lighthouse laser system HTC Vive IR LED constellation tracking system Oculus Rift ifixit Inside-out

More information

Very High Frame Rate Volumetric Integration of Depth Images on Mobile Devices

Very High Frame Rate Volumetric Integration of Depth Images on Mobile Devices Very High Frame Rate Volumetric Integration of Depth Images on Mobile Devices Olaf Kähler*, Victor Adrian Prisacariu*, Carl Yuheng Ren, Xin Sun, Philip Torr, David Murray Fig. 1: Sample of the reconstruction

More information

Volumetric 3D Mapping in Real-Time on a CPU

Volumetric 3D Mapping in Real-Time on a CPU Volumetric 3D Mapping in Real-Time on a CPU Frank Steinbrücker, Jürgen Sturm, and Daniel Cremers Abstract In this paper we propose a novel volumetric multi-resolution mapping system for RGB-D images that

More information

Building a 3D Object Scanner

Building a 3D Object Scanner Distributed Computing Building a 3D Object Scanner Bachelor Thesis Ulla Aeschbacher ullaa@ethz.ch Distributed Computing Group Computer Engineering and Networks Laboratory ETH Zürich Supervisors: Manuel

More information

Real-Time Vision-Based State Estimation and (Dense) Mapping

Real-Time Vision-Based State Estimation and (Dense) Mapping Real-Time Vision-Based State Estimation and (Dense) Mapping Stefan Leutenegger IROS 2016 Workshop on State Estimation and Terrain Perception for All Terrain Mobile Robots The Perception-Action Cycle in

More information

SPURRED by the ready availability of depth sensors and

SPURRED by the ready availability of depth sensors and IEEE ROBOTICS AND AUTOMATION LETTERS. PREPRINT VERSION. ACCEPTED DECEMBER, 2015 1 Hierarchical Hashing for Efficient Integration of Depth Images Olaf Kähler, Victor Prisacariu, Julien Valentin and David

More information

What s New in Emulate3D. New Tag Grid

What s New in Emulate3D. New Tag Grid What s New in 2016 Emulate3D New Tag Grid The TagBrowser Tag Grid now appears in the main Demo3D properties pane, making it easier to see the tags bound to each component in the model, and to observe component

More information

FLaME: Fast Lightweight Mesh Estimation using Variational Smoothing on Delaunay Graphs

FLaME: Fast Lightweight Mesh Estimation using Variational Smoothing on Delaunay Graphs FLaME: Fast Lightweight Mesh Estimation using Variational Smoothing on Delaunay Graphs W. Nicholas Greene Robust Robotics Group, MIT CSAIL LPM Workshop IROS 2017 September 28, 2017 with Nicholas Roy 1

More information

Mobile Point Fusion. Real-time 3d surface reconstruction out of depth images on a mobile platform

Mobile Point Fusion. Real-time 3d surface reconstruction out of depth images on a mobile platform Mobile Point Fusion Real-time 3d surface reconstruction out of depth images on a mobile platform Aaron Wetzler Presenting: Daniel Ben-Hoda Supervisors: Prof. Ron Kimmel Gal Kamar Yaron Honen Supported

More information

Fast Interactive Region of Interest Selection for Volume Visualization

Fast Interactive Region of Interest Selection for Volume Visualization Fast Interactive Region of Interest Selection for Volume Visualization Dominik Sibbing and Leif Kobbelt Lehrstuhl für Informatik 8, RWTH Aachen, 20 Aachen Email: {sibbing,kobbelt}@informatik.rwth-aachen.de

More information

A Volumetric Method for Building Complex Models from Range Images

A Volumetric Method for Building Complex Models from Range Images A Volumetric Method for Building Complex Models from Range Images 1. Briefly summarize the paper s contributions. Does it address a new problem? Does it present a new approach? Does it show new types of

More information