Exploiting Depth Camera for 3D Spatial Relationship Interpretation
|
|
- Hope Conley
- 6 years ago
- Views:
Transcription
1 Exploiting Depth Camera for 3D Spatial Relationship Interpretation Jun Ye Kien A. Hua Data Systems Group, University of Central Florida Mar 1, 2013 Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
2 Outline Motivation, definition and background Define 3D directional relationships 3D scene reconstruction Compute 3D directional relationships between two objects Performance study Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
3 Definition Definition: spatial relationship specifies how an object is located w.r.t. other objects. Applications: robotic spatial reasoning scene understanding video surveillance, spatial query. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
4 Spatial relationships Topological relationships coincide, intersect, touch externally, touch internally, contains, inside, outside, disjoint, etc. Metric relationships Distance relationships at, nearby, far away from, etc. Directional relationships east, west, north, south, northeast, northwest, southeast, southwest, etc. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
5 Limitation of 2D spatial representation sitting on a chair (contact) standing in front of a chair (no contact) Ambiguity 2D spatial representation cannot distinguish these two spatial relations. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
6 Extension from 2D to 3D Device Kinect sensor (depth sensor) RGB image + depth image Spatial direction set Define a new set of 3D directional relationships Algorithm for computing the new relationships New method leverages the 3D object model Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
7 Application background We investigate the 3D spatial relationships representation as a component for our Live Video DataBase Management System (LVDBMS) [MMSJ 12]. The LVDBMS is a general-purpose framework for managing and processing video data for surveillance and analytical applications. This system allows automatic monitoring and management of a network of live cameras. The user specifies a monitoring task by formulating a query to describe a spatiotemporal event. The query is in the form of combinations of some logical, spatial, and temporal operators. When the specified event occurs, an action associated with the query is triggered. The LVDBMS treats a camera as a special class of storage, and processes the queries against the live video feed as a new category of databases. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
8 Outline Motivation, definition and background Define 3D directional relationships 3D scene reconstruction Compute 3D directional relationships between two objects Performance study Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
9 Define 3D directional relationships: 2D to 3D extension 2D spatial direction is defined in a plane 3D spatial direction is define in space by introducing another two directions, above and below By combining the original 8 2D directions with A (above) and B (below), 26 primitive 3D directions are defined. They can be fit to a 3x3 cube. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
10 Define 3D directional relationships: a complete set of 26 directions in 3D space There 27 grids in the 3x3 cube, supposing the reference object locates at the grid in the center. The 26 3D directions can be presented in the following way. ASW AS ASE AE ANE AN ANW AW SW S SE E NE N NW W BSW BS BSE BE BNE BN BNW BW A B 26 primitive 3D directions can be nicely fit into a 3X 3 cube. e.g. ASE means southeast and to the above Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
11 Outline Motivation, definition and background Define 3D directional relationships 3D scene reconstruction Compute 3D directional relationships between two objects Performance study Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
12 3D reconstruction: 3D point cloud Pixel in RGB image, 3 channels, 24 bits (i, j, r, g, b) Pixel in depth image, 1 channel, 16 bits (i, j, z) Point in 3D point cloud (x, y, z, r, g, b) x = (i u0) z a x y = (j v0) z a y (u 0, v 0 ): principle point, a x, a y : focal length Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
13 3D reconstruction: compute the camera pose Estimate the floor plane by RANSAC algorithm Compute the camera pose using the normal of floor plane Rotate the 3D point cloud according to camera pose 3D point cloud is then aligned to the world coordinate system Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
14 3D reconstruction: extract objects User specify the reference obj and the target obj Stationary obj (e.g. chair, table) are manually specified and segmentated Meanshift segmentation Filter background pixels (floor, walls, etc) using RGB image Extract object using depth image Mobile object (mostly human beings), automatically detected and tracked Using Kinect SDK Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
15 3D reconstruction: fill up the model by heuristics Single view only capture half information of the scene 3D point cloud is incomplete, back side point is missing The thickness of objects may be incorrectly represented thus leads to incorrect spatial relationships (e.g. side by side or behind?) Fill up the 3D model by heuristics (Manhattan suggestion), assume the back side pixel has the same depth as pixel on top, a vertical projection from top view Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
16 Outline Motivation, definition and background Define 3D directional relationships 3D scene reconstruction Compute 3D directional relationships between two objects Performance study Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
17 Compute 3D directions: sample the sphere The idea is adopted from ray tracing (computer graphics techniques for real-time rendering) 840 rays are originated from the reference point and go into the space at different directions. Each ray represents a particular direction < θ, φ > under polar coordinate system θ and φ are discretized into 21 and 40 sample points, respectively. θ [0, π], φ [0, 2π), step = π 20 Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
18 Compute 3D directions Each ray will go through the space until hit an object (parts of the object). It produces a result of 0 or 1, with 1 indicating a hit. A 840-dimension feature vector Advantage Since rays sample the entirely space around the reference object, it automatically investigate both the position and shape information of the target object. It can handle object with complex shapes. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
19 Compute 3D directions: mapping to 26 directions Convert the 840D featrue vector to the 26 premitive directions. In accordance with the 3X3 cube holding the 26 premitive directions. Clustering neighboring rays into primitive directions by quantizing θ and φ into 5 and 8 sub-divisions. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
20 Compute 3D directions: mapping to 26 directions (cont d) Each primitive direction covers 25 rays (5 consecutive θ values and φ values) They have different significance (ray through the center part are more significant) 25 rays are weighted by a 5x5 Gaussian filter g(x, y) = 1 A 1 2πσ e x 2 +y 2 2 2σ 2 Weighted sum of all 25 rays results = 2 2 f(x, y)g(x, y), x= 2 y= 2 { 1, ray hits the obj, f(x, y) = 0, otherwise 840D bool feature vector is converted to the 26D float feature vector Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
21 Compute 3D directions: Octree decomposition From which point of the reference object do we generate rays? cannot include all points, too computational intense a small object can have more than 50,000 points in the point cloud. Octree coding is used to reduce the model while preserving enough structural information. An Octree is a data structure that each node has 8 child nodes. It can recursively subdivide a cube into 8 sub-cubicles. n levels of Octree decomposition partitions the space into 8 n sub-cubicles. Starting from the bounding box of the object and subdivide it. Remove those subcubicles do not overlap with the object. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
22 Compute 3D directions between two objects Centroid-based method Generate 840 rays from the centroid of the reference object Check if each ray intersects with the point cloud of the target object Compute the 26D float features Sort the results in descending order and output results greater than zero Simple, fast, useful when target obj is significantly larger than the reference obj. Landmark-based method 2 levels of Octree decomposition is performed to the bounding box of the reference object Do the same thing at the centroid of each subcubicles Accumulate the results of each subcubicles Robust to spatial relationships between complex object, but heavy on computation. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
23 Outline Motivation, definition and background Define 3D directional relationships 3D scene reconstruction Compute 3D directional relationships between two objects Performance study Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
24 Performance study We perform the experimental study on both a real indoor environment, an office scene objects include chair, table, etc RGB and Depth images a public available RGB-D dataset contain 300 common household objects (bowls, cups, dishes, computers, etc) synchronized RGB and Depth images sequences captured from 8 different scenes Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
25 Performance study: demonstration in real scene Demonstration in an indoor environment (a) (b) (c) (d) Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
26 Performance study: experiments on RGB-D dataset 43 representative images are selected from the total image sequences Since no ground truth (GT) is provided, we manually label the image by ourselves. 214 pairs of objects are labeled from the 43 images and 364 spatial relationships between them are labeled. There could be multiple spatial relationships between a pair of objects. We annotate the most salient one as the major relationships and all others (include the major relationship) as candidate relationships. It is extremely challenging to label the GT perfectly. Since spatial relationship is a fuzzy concept, we label it in accordance with human perception. Experiments are performed based on both major relationships and candidate relationships Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
27 Performance study: experiments on RGB-D dataset Performance evaluation based on major relationship Centroid-based method Landmark-based method 89% of the object pairs are correctly detected regarding to the major relationship, our algorithm suits human perception Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
28 Performance study: experiments on RGB-D dataset Performance evaluation based on all candidate relationship, recall and precision are applied as the performance metrics, recall = precision = Centroid-based method num of correctly detected relationships total num of relationships in the GT, num of correctly detected relationships num of detected relationships Landmark-based method Landmark-based method significantly increase the recall without sacrificing too much precision. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
29 Performance study: experiments on RGB-D dataset Performance based on speed. The elapsed time only includes the 3D object modeling and the spatial relationship detection Hardware: Intel i7 quad cores CPU and 24GB RAM. Landmark-based method repeat the same process for n times (n = number of subcubicles), it is n times slower than centroid method GPU techniques can be exploited to accelerate the algorithm, e.g. OpenCL, CUDA. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
30 Conclusions We define a new set of 3D directional spatial relationships We introduce an efficient algorithm to compute the above 3D spatial relationships Extensive experimental study based on a real scene and a public dataset demonstrate the effectiveness of our algorithm. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
31 Discussion and future work Geodesic sphere ray generation using polar coordinate system results more ray density at direction of two poles of a sphere geodesic sphere equally devides the surface of a spshere GPU implementation OpenCL, CUDA, etc. fully parallel, each OpenCL kernel process an octree subcubicle. Better way other than heuristics multiple kinects one moving kinect: kinect fusion [SIGGRAPH 2011] better heuristics Push Octree model ahead represent the 3D model by octree rather than pointcloud segmentation on Octree compute spatial relationship entirely based on octree Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
32 Thank you. Jun Ye and Kien A. Hua (UCF) 3D directional spatial relationships Mar 1, / 32
Intersection Acceleration
Advanced Computer Graphics Intersection Acceleration Matthias Teschner Computer Science Department University of Freiburg Outline introduction bounding volume hierarchies uniform grids kd-trees octrees
More informationCS395T paper review. Indoor Segmentation and Support Inference from RGBD Images. Chao Jia Sep
CS395T paper review Indoor Segmentation and Support Inference from RGBD Images Chao Jia Sep 28 2012 Introduction What do we want -- Indoor scene parsing Segmentation and labeling Support relationships
More informationAcceleration Data Structures
CT4510: Computer Graphics Acceleration Data Structures BOCHANG MOON Ray Tracing Procedure for Ray Tracing: For each pixel Generate a primary ray (with depth 0) While (depth < d) { Find the closest intersection
More informationAccelerating Geometric Queries. Computer Graphics CMU /15-662, Fall 2016
Accelerating Geometric Queries Computer Graphics CMU 15-462/15-662, Fall 2016 Geometric modeling and geometric queries p What point on the mesh is closest to p? What point on the mesh is closest to p?
More informationSemantic-Based Surveillance Video Retrieval
Semantic-Based Surveillance Video Retrieval Weiming Hu, Dan Xie, Zhouyu Fu, Wenrong Zeng, and Steve Maybank, Senior Member, IEEE IEEE Transactions on Image Processing, Vol. 16, No. 4, April 2007 Present
More informationIdentifying those parts of a scene that are visible from a chosen viewing position, and only process (scan convert) those parts
Visible Surface Detection Identifying those parts of a scene that are visible from a chosen viewing position, and only process (scan convert) those parts Two approaches: 1. Object space methods 2. Image
More informationINFOGR Computer Graphics. J. Bikker - April-July Lecture 11: Acceleration. Welcome!
INFOGR Computer Graphics J. Bikker - April-July 2015 - Lecture 11: Acceleration Welcome! Today s Agenda: High-speed Ray Tracing Acceleration Structures The Bounding Volume Hierarchy BVH Construction BVH
More informationL10 Layered Depth Normal Images. Introduction Related Work Structured Point Representation Boolean Operations Conclusion
L10 Layered Depth Normal Images Introduction Related Work Structured Point Representation Boolean Operations Conclusion 1 Introduction Purpose: using the computational power on GPU to speed up solid modeling
More informationDense Tracking and Mapping for Autonomous Quadrocopters. Jürgen Sturm
Computer Vision Group Prof. Daniel Cremers Dense Tracking and Mapping for Autonomous Quadrocopters Jürgen Sturm Joint work with Frank Steinbrücker, Jakob Engel, Christian Kerl, Erik Bylow, and Daniel Cremers
More informationCPSC / Sonny Chan - University of Calgary. Collision Detection II
CPSC 599.86 / 601.86 Sonny Chan - University of Calgary Collision Detection II Outline Broad phase collision detection: - Problem definition and motivation - Bounding volume hierarchies - Spatial partitioning
More informationRay Intersection Acceleration
Ray Intersection Acceleration CMPT 461/761 Image Synthesis Torsten Möller Reading Chapter 2, 3, 4 of Physically Based Rendering by Pharr&Humphreys An Introduction to Ray tracing by Glassner Topics today
More informationAccelerating Ray Tracing
Accelerating Ray Tracing Ray Tracing Acceleration Techniques Faster Intersections Fewer Rays Generalized Rays Faster Ray-Object Intersections Object bounding volumes Efficient intersection routines Fewer
More informationMesh Generation. Quadtrees. Geometric Algorithms. Lecture 9: Quadtrees
Lecture 9: Lecture 9: VLSI Design To Lecture 9: Finite Element Method To http://www.antics1.demon.co.uk/finelms.html Lecture 9: To Lecture 9: To component not conforming doesn t respect input not well-shaped
More informationSpatial Data Structures and Speed-Up Techniques. Tomas Akenine-Möller Department of Computer Engineering Chalmers University of Technology
Spatial Data Structures and Speed-Up Techniques Tomas Akenine-Möller Department of Computer Engineering Chalmers University of Technology Spatial data structures What is it? Data structure that organizes
More informationArticulated Pose Estimation with Flexible Mixtures-of-Parts
Articulated Pose Estimation with Flexible Mixtures-of-Parts PRESENTATION: JESSE DAVIS CS 3710 VISUAL RECOGNITION Outline Modeling Special Cases Inferences Learning Experiments Problem and Relevance Problem:
More informationRay Tracing Acceleration Data Structures
Ray Tracing Acceleration Data Structures Sumair Ahmed October 29, 2009 Ray Tracing is very time-consuming because of the ray-object intersection calculations. With the brute force method, each ray has
More informationPage 1. Area-Subdivision Algorithms z-buffer Algorithm List Priority Algorithms BSP (Binary Space Partitioning Tree) Scan-line Algorithms
Visible Surface Determination Visibility Culling Area-Subdivision Algorithms z-buffer Algorithm List Priority Algorithms BSP (Binary Space Partitioning Tree) Scan-line Algorithms Divide-and-conquer strategy:
More informationExtracting Layers and Recognizing Features for Automatic Map Understanding. Yao-Yi Chiang
Extracting Layers and Recognizing Features for Automatic Map Understanding Yao-Yi Chiang 0 Outline Introduction/ Problem Motivation Map Processing Overview Map Decomposition Feature Recognition Discussion
More informationReal-Time Ray Tracing Using Nvidia Optix Holger Ludvigsen & Anne C. Elster 2010
1 Real-Time Ray Tracing Using Nvidia Optix Holger Ludvigsen & Anne C. Elster 2010 Presentation by Henrik H. Knutsen for TDT24, fall 2012 Om du ønsker, kan du sette inn navn, tittel på foredraget, o.l.
More informationRay Tracing. Computer Graphics CMU /15-662, Fall 2016
Ray Tracing Computer Graphics CMU 15-462/15-662, Fall 2016 Primitive-partitioning vs. space-partitioning acceleration structures Primitive partitioning (bounding volume hierarchy): partitions node s primitives
More informationCollision Detection based on Spatial Partitioning
Simulation in Computer Graphics Collision Detection based on Spatial Partitioning Matthias Teschner Computer Science Department University of Freiburg Outline introduction uniform grid Octree and k-d tree
More informationFundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision
Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision What Happened Last Time? Human 3D perception (3D cinema) Computational stereo Intuitive explanation of what is meant by disparity Stereo matching
More informationCS 4758: Automated Semantic Mapping of Environment
CS 4758: Automated Semantic Mapping of Environment Dongsu Lee, ECE, M.Eng., dl624@cornell.edu Aperahama Parangi, CS, 2013, alp75@cornell.edu Abstract The purpose of this project is to program an Erratic
More informationNear Real-time Pointcloud Smoothing for 3D Imaging Systems Colin Lea CS420 - Parallel Programming - Fall 2011
Near Real-time Pointcloud Smoothing for 3D Imaging Systems Colin Lea CS420 - Parallel Programming - Fall 2011 Abstract -- In this project a GPU-based implementation of Moving Least Squares is presented
More informationTopics. Ray Tracing II. Intersecting transformed objects. Transforming objects
Topics Ray Tracing II CS 4620 Lecture 16 Transformations in ray tracing Transforming objects Transformation hierarchies Ray tracing acceleration structures Bounding volumes Bounding volume hierarchies
More informationFast Fuzzy Clustering of Infrared Images. 2. brfcm
Fast Fuzzy Clustering of Infrared Images Steven Eschrich, Jingwei Ke, Lawrence O. Hall and Dmitry B. Goldgof Department of Computer Science and Engineering, ENB 118 University of South Florida 4202 E.
More informationGeometric Structures 2. Quadtree, k-d stromy
Geometric Structures 2. Quadtree, k-d stromy Martin Samuelčík samuelcik@sccg.sk, www.sccg.sk/~samuelcik, I4 Window and Point query From given set of points, find all that are inside of given d-dimensional
More informationMotion Tracking and Event Understanding in Video Sequences
Motion Tracking and Event Understanding in Video Sequences Isaac Cohen Elaine Kang, Jinman Kang Institute for Robotics and Intelligent Systems University of Southern California Los Angeles, CA Objectives!
More informationMobile Point Fusion. Real-time 3d surface reconstruction out of depth images on a mobile platform
Mobile Point Fusion Real-time 3d surface reconstruction out of depth images on a mobile platform Aaron Wetzler Presenting: Daniel Ben-Hoda Supervisors: Prof. Ron Kimmel Gal Kamar Yaron Honen Supported
More informationSimulation in Computer Graphics Space Subdivision. Matthias Teschner
Simulation in Computer Graphics Space Subdivision Matthias Teschner Outline Introduction Uniform grid Octree and k-d tree BSP tree University of Freiburg Computer Science Department 2 Model Partitioning
More informationDiFi: Distance Fields - Fast Computation Using Graphics Hardware
DiFi: Distance Fields - Fast Computation Using Graphics Hardware Avneesh Sud Dinesh Manocha UNC-Chapel Hill http://gamma.cs.unc.edu/difi Distance Fields Distance Function For a site a scalar function f:r
More informationPoint Cloud Filtering using Ray Casting by Eric Jensen 2012 The Basic Methodology
Point Cloud Filtering using Ray Casting by Eric Jensen 01 The Basic Methodology Ray tracing in standard graphics study is a method of following the path of a photon from the light source to the camera,
More informationHeight Filed Simulation and Rendering
Height Filed Simulation and Rendering Jun Ye Data Systems Group, University of Central Florida April 18, 2013 Jun Ye (UCF) Height Filed Simulation and Rendering April 18, 2013 1 / 20 Outline Three primary
More informationL1 - Introduction. Contents. Introduction of CAD/CAM system Components of CAD/CAM systems Basic concepts of graphics programming
L1 - Introduction Contents Introduction of CAD/CAM system Components of CAD/CAM systems Basic concepts of graphics programming 1 Definitions Computer-Aided Design (CAD) The technology concerned with the
More informationRay Tracing with Spatial Hierarchies. Jeff Mahovsky & Brian Wyvill CSC 305
Ray Tracing with Spatial Hierarchies Jeff Mahovsky & Brian Wyvill CSC 305 Ray Tracing Flexible, accurate, high-quality rendering Slow Simplest ray tracer: Test every ray against every object in the scene
More informationImproving Memory Space Efficiency of Kd-tree for Real-time Ray Tracing Byeongjun Choi, Byungjoon Chang, Insung Ihm
Improving Memory Space Efficiency of Kd-tree for Real-time Ray Tracing Byeongjun Choi, Byungjoon Chang, Insung Ihm Department of Computer Science and Engineering Sogang University, Korea Improving Memory
More informationComputer Graphics. Bing-Yu Chen National Taiwan University The University of Tokyo
Computer Graphics Bing-Yu Chen National Taiwan University The University of Tokyo Hidden-Surface Removal Back-Face Culling The Depth-Sort Algorithm Binary Space-Partitioning Trees The z-buffer Algorithm
More informationToday. Acceleration Data Structures for Ray Tracing. Cool results from Assignment 2. Last Week: Questions? Schedule
Today Acceleration Data Structures for Ray Tracing Cool results from Assignment 2 Last Week: koi seantek Ray Tracing Shadows Reflection Refraction Local Illumination Bidirectional Reflectance Distribution
More informationScene Management. Video Game Technologies 11498: MSc in Computer Science and Engineering 11156: MSc in Game Design and Development
Video Game Technologies 11498: MSc in Computer Science and Engineering 11156: MSc in Game Design and Development Chap. 5 Scene Management Overview Scene Management vs Rendering This chapter is about rendering
More informationThree-Dimensional Object Detection and Layout Prediction using Clouds of Oriented Gradients
ThreeDimensional Object Detection and Layout Prediction using Clouds of Oriented Gradients Authors: Zhile Ren, Erik B. Sudderth Presented by: Shannon Kao, Max Wang October 19, 2016 Introduction Given an
More informationRay Tracing. Kjetil Babington
Ray Tracing Kjetil Babington 21.10.2011 1 Introduction What is Ray Tracing? Act of tracing a ray through some scene Not necessarily for rendering Rendering with Ray Tracing Ray Tracing is a global illumination
More informationRay Tracing Acceleration. CS 4620 Lecture 20
Ray Tracing Acceleration CS 4620 Lecture 20 2013 Steve Marschner 1 Will this be on the exam? or, Prelim 2 syllabus You can expect emphasis on topics related to the assignment (Shaders 1&2) and homework
More informationA Low Power, High Throughput, Fully Event-Based Stereo System: Supplementary Documentation
A Low Power, High Throughput, Fully Event-Based Stereo System: Supplementary Documentation Alexander Andreopoulos, Hirak J. Kashyap, Tapan K. Nayak, Arnon Amir, Myron D. Flickner IBM Research March 25,
More informationSeparating Objects and Clutter in Indoor Scenes
Separating Objects and Clutter in Indoor Scenes Salman H. Khan School of Computer Science & Software Engineering, The University of Western Australia Co-authors: Xuming He, Mohammed Bennamoun, Ferdous
More informationTopics. Ray Tracing II. Transforming objects. Intersecting transformed objects
Topics Ray Tracing II CS 4620 ations in ray tracing ing objects ation hierarchies Ray tracing acceleration structures Bounding volumes Bounding volume hierarchies Uniform spatial subdivision Adaptive spatial
More informationInteractive Collision Detection for Engineering Plants based on Large-Scale Point-Clouds
1 Interactive Collision Detection for Engineering Plants based on Large-Scale Point-Clouds Takeru Niwa 1 and Hiroshi Masuda 2 1 The University of Electro-Communications, takeru.niwa@uec.ac.jp 2 The University
More informationELEN E4830 Digital Image Processing. Homework 6 Solution
ELEN E4830 Digital Image Processing Homework 6 Solution Chuxiang Li cxli@ee.columbia.edu Department of Electrical Engineering, Columbia University April 10, 2006 1 Edge Detection 1.1 Sobel Operator The
More informationA physically motivated pixel-based model for background subtraction in 3D images
A physically motivated pixel-based model for background subtraction in 3D images M. Braham, A. Lejeune and M. Van Droogenbroeck INTELSIG, Montefiore Institute, University of Liège, Belgium IC3D - December
More informationLecture: Autonomous micro aerial vehicles
Lecture: Autonomous micro aerial vehicles Friedrich Fraundorfer Remote Sensing Technology TU München 1/41 Autonomous operation@eth Zürich Start 2/41 Autonomous operation@eth Zürich 3/41 Outline MAV system
More informationSpeeding Up Ray Tracing. Optimisations. Ray Tracing Acceleration
Speeding Up Ray Tracing nthony Steed 1999, eline Loscos 2005, Jan Kautz 2007-2009 Optimisations Limit the number of rays Make the ray test faster for shadow rays the main drain on resources if there are
More informationCSG obj. oper3. obj1 obj2 obj3. obj5. obj4
Solid Modeling Solid: Boundary + Interior Volume occupied by geometry Solid representation schemes Constructive Solid Geometry (CSG) Boundary representations (B-reps) Space-partition representations Operations
More informationRay-tracing Acceleration. Acceleration Data Structures for Ray Tracing. Shadows. Shadows & Light Sources. Antialiasing Supersampling.
Ray-tracing Acceleration Acceleration Data Structures for Ray Tracing Thanks to Fredo Durand and Barb Cutler Soft shadows Antialiasing (getting rid of jaggies) Glossy reflection Motion blur Depth of field
More informationComputer Graphics. Bing-Yu Chen National Taiwan University
Computer Graphics Bing-Yu Chen National Taiwan University Visible-Surface Determination Back-Face Culling The Depth-Sort Algorithm Binary Space-Partitioning Trees The z-buffer Algorithm Scan-Line Algorithm
More informationSpatial Data Structures
15-462 Computer Graphics I Lecture 17 Spatial Data Structures Hierarchical Bounding Volumes Regular Grids Octrees BSP Trees Constructive Solid Geometry (CSG) April 1, 2003 [Angel 9.10] Frank Pfenning Carnegie
More informationRay Tracing: Intersection
Computer Graphics as Virtual Photography Ray Tracing: Intersection Photography: real scene camera (captures light) photo processing Photographic print processing Computer Graphics: 3D models camera tone
More informationCPSC GLOBAL ILLUMINATION
CPSC 314 21 GLOBAL ILLUMINATION Textbook: 20 UGRAD.CS.UBC.CA/~CS314 Mikhail Bessmeltsev ILLUMINATION MODELS/ALGORITHMS Local illumination - Fast Ignore real physics, approximate the look Interaction of
More informationSpatial Data Structures
Spatial Data Structures Hierarchical Bounding Volumes Regular Grids Octrees BSP Trees Constructive Solid Geometry (CSG) [Angel 9.10] Outline Ray tracing review what rays matter? Ray tracing speedup faster
More informationPerception. Autonomous Mobile Robots. Sensors Vision Uncertainties, Line extraction from laser scans. Autonomous Systems Lab. Zürich.
Autonomous Mobile Robots Localization "Position" Global Map Cognition Environment Model Local Map Path Perception Real World Environment Motion Control Perception Sensors Vision Uncertainties, Line extraction
More informationTopic 10: Scene Management, Particle Systems and Normal Mapping. CITS4242: Game Design and Multimedia
CITS4242: Game Design and Multimedia Topic 10: Scene Management, Particle Systems and Normal Mapping Scene Management Scene management means keeping track of all objects in a scene. - In particular, keeping
More informationData Association for SLAM
CALIFORNIA INSTITUTE OF TECHNOLOGY ME/CS 132a, Winter 2011 Lab #2 Due: Mar 10th, 2011 Part I Data Association for SLAM 1 Introduction For this part, you will experiment with a simulation of an EKF SLAM
More informationAccelerating Ray-Tracing
Lecture 9: Accelerating Ray-Tracing Computer Graphics and Imaging UC Berkeley CS184/284A, Spring 2016 Course Roadmap Rasterization Pipeline Core Concepts Sampling Antialiasing Transforms Geometric Modeling
More informationEfficient Depth-Compensated Interpolation for Full Parallax Displays
ETN-FPI TS3 Plenoptic Sensing Efficient Depth-Compensated Interpolation for Full Parallax Displays Reinhard Koch and Daniel Jung Multimedia Information Processing Full parallax autostereoscopic display
More informationCS 4620 Midterm, March 21, 2017
CS 460 Midterm, March 1, 017 This 90-minute exam has 4 questions worth a total of 100 points. Use the back of the pages if you need more space. Academic Integrity is expected of all students of Cornell
More information08 An Introduction to Dense Continuous Robotic Mapping
NAVARCH/EECS 568, ROB 530 - Winter 2018 08 An Introduction to Dense Continuous Robotic Mapping Maani Ghaffari March 14, 2018 Previously: Occupancy Grid Maps Pose SLAM graph and its associated dense occupancy
More informationCS488. Visible-Surface Determination. Luc RENAMBOT
CS488 Visible-Surface Determination Luc RENAMBOT 1 Visible-Surface Determination So far in the class we have dealt mostly with simple wireframe drawings of the models The main reason for this is so that
More informationLearning Semantic Environment Perception for Cognitive Robots
Learning Semantic Environment Perception for Cognitive Robots Sven Behnke University of Bonn, Germany Computer Science Institute VI Autonomous Intelligent Systems Some of Our Cognitive Robots Equipped
More informationRay Tracing. CS334 Fall Daniel G. Aliaga Department of Computer Science Purdue University
Ray Tracing CS334 Fall 2013 Daniel G. Aliaga Department of Computer Science Purdue University Ray Casting and Ray Tracing Ray Casting Arthur Appel, started around 1968 Ray Tracing Turner Whitted, started
More informationSpatial Data Structures
15-462 Computer Graphics I Lecture 17 Spatial Data Structures Hierarchical Bounding Volumes Regular Grids Octrees BSP Trees Constructive Solid Geometry (CSG) March 28, 2002 [Angel 8.9] Frank Pfenning Carnegie
More informationMorphological track 1
Morphological track 1 Shapes Painting of living beings on cave walls at Lascaux [about 1500 th BC] L homme qui marche by Alberto Giacometti, 1948, NOUVELLES IMAGES Editor (1976) Les lutteurs by Honoré
More informationReal-Time Voxelization for Global Illumination
Lecture 26: Real-Time Voxelization for Global Illumination Visual Computing Systems Voxelization to regular grid Input: scene triangles Output: surface information at each voxel in 3D grid - Simple case:
More informationSpatial Data Structures
CSCI 420 Computer Graphics Lecture 17 Spatial Data Structures Jernej Barbic University of Southern California Hierarchical Bounding Volumes Regular Grids Octrees BSP Trees [Angel Ch. 8] 1 Ray Tracing Acceleration
More informationRay Tracing III. Wen-Chieh (Steve) Lin National Chiao-Tung University
Ray Tracing III Wen-Chieh (Steve) Lin National Chiao-Tung University Shirley, Fundamentals of Computer Graphics, Chap 10 Doug James CG slides, I-Chen Lin s CG slides Ray-tracing Review For each pixel,
More informationLecture 7: Most Common Edge Detectors
#1 Lecture 7: Most Common Edge Detectors Saad Bedros sbedros@umn.edu Edge Detection Goal: Identify sudden changes (discontinuities) in an image Intuitively, most semantic and shape information from the
More informationA Real-time Algorithm for Atmospheric Turbulence Correction
Logic Fruit Technologies White Paper 806, 8 th Floor, BPTP Park Centra, Sector 30, Gurgaon. Pin: 122001 T: +91-124-4117336 W: http://www.logic-fruit.com A Real-time Algorithm for Atmospheric Turbulence
More information1. Meshes. D7013E Lecture 14
D7013E Lecture 14 Quadtrees Mesh Generation 1. Meshes Input: Components in the form of disjoint polygonal objects Integer coordinates, 0, 45, 90, or 135 angles Output: A triangular mesh Conforming: A triangle
More informationRobotics Programming Laboratory
Chair of Software Engineering Robotics Programming Laboratory Bertrand Meyer Jiwon Shin Lecture 8: Robot Perception Perception http://pascallin.ecs.soton.ac.uk/challenges/voc/databases.html#caltech car
More informationCAMERA POSE ESTIMATION OF RGB-D SENSORS USING PARTICLE FILTERING
CAMERA POSE ESTIMATION OF RGB-D SENSORS USING PARTICLE FILTERING By Michael Lowney Senior Thesis in Electrical Engineering University of Illinois at Urbana-Champaign Advisor: Professor Minh Do May 2015
More informationLecture 25 of 41. Spatial Sorting: Binary Space Partitioning Quadtrees & Octrees
Spatial Sorting: Binary Space Partitioning Quadtrees & Octrees William H. Hsu Department of Computing and Information Sciences, KSU KSOL course pages: http://bit.ly/hgvxlh / http://bit.ly/evizre Public
More informationReconstruction of Polygonal Faces from Large-Scale Point-Clouds of Engineering Plants
1 Reconstruction of Polygonal Faces from Large-Scale Point-Clouds of Engineering Plants Hiroshi Masuda 1, Takeru Niwa 2, Ichiro Tanaka 3 and Ryo Matsuoka 4 1 The University of Electro-Communications, h.masuda@euc.ac.jp
More informationAdvances in 3D data processing and 3D cameras
Advances in 3D data processing and 3D cameras Miguel Cazorla Grupo de Robótica y Visión Tridimensional Universidad de Alicante Contents Cameras and 3D images 3D data compression 3D registration 3D feature
More informationSpatial Data Structures
CSCI 480 Computer Graphics Lecture 7 Spatial Data Structures Hierarchical Bounding Volumes Regular Grids BSP Trees [Ch. 0.] March 8, 0 Jernej Barbic University of Southern California http://www-bcf.usc.edu/~jbarbic/cs480-s/
More informationCity-Modeling. Detecting and Reconstructing Buildings from Aerial Images and LIDAR Data
City-Modeling Detecting and Reconstructing Buildings from Aerial Images and LIDAR Data Department of Photogrammetrie Institute for Geodesy and Geoinformation Bonn 300000 inhabitants At river Rhine University
More informationSolid Modeling. Ron Goldman Department of Computer Science Rice University
Solid Modeling Ron Goldman Department of Computer Science Rice University Solids Definition 1. A model which has a well defined inside and outside. 2. For each point, we can in principle determine whether
More informationLecture 11: Ray tracing (cont.)
Interactive Computer Graphics Ray tracing - Summary Lecture 11: Ray tracing (cont.) Graphics Lecture 10: Slide 1 Some slides adopted from H. Pfister, Harvard Graphics Lecture 10: Slide 2 Ray tracing -
More informationAccurate 3D Face and Body Modeling from a Single Fixed Kinect
Accurate 3D Face and Body Modeling from a Single Fixed Kinect Ruizhe Wang*, Matthias Hernandez*, Jongmoo Choi, Gérard Medioni Computer Vision Lab, IRIS University of Southern California Abstract In this
More informationHello, Thanks for the introduction
Hello, Thanks for the introduction 1 In this paper we suggest an efficient data-structure for precomputed shadows from point light or directional light-sources. Because, in fact, after more than four decades
More information3D Reconstruction with Tango. Ivan Dryanovski, Google Inc.
3D Reconstruction with Tango Ivan Dryanovski, Google Inc. Contents Problem statement and motivation The Tango SDK 3D reconstruction - data structures & algorithms Applications Developer tools Problem formulation
More informationCS 465 Program 5: Ray II
CS 465 Program 5: Ray II out: Friday 2 November 2007 due: Saturday 1 December 2007 Sunday 2 December 2007 midnight 1 Introduction In the first ray tracing assignment you built a simple ray tracer that
More informationUncertainties: Representation and Propagation & Line Extraction from Range data
41 Uncertainties: Representation and Propagation & Line Extraction from Range data 42 Uncertainty Representation Section 4.1.3 of the book Sensing in the real world is always uncertain How can uncertainty
More informationLast Time: Acceleration Data Structures for Ray Tracing. Schedule. Today. Shadows & Light Sources. Shadows
Last Time: Acceleration Data Structures for Ray Tracing Modeling Transformations Illumination (Shading) Viewing Transformation (Perspective / Orthographic) Clipping Projection (to Screen Space) Scan Conversion
More informationCRF Based Point Cloud Segmentation Jonathan Nation
CRF Based Point Cloud Segmentation Jonathan Nation jsnation@stanford.edu 1. INTRODUCTION The goal of the project is to use the recently proposed fully connected conditional random field (CRF) model to
More informationCS535 Fall Department of Computer Science Purdue University
Spatial Data Structures and Hierarchies CS535 Fall 2010 Daniel G Aliaga Daniel G. Aliaga Department of Computer Science Purdue University Spatial Data Structures Store geometric information Organize geometric
More informationGeometric Algorithms. Geometric search: overview. 1D Range Search. 1D Range Search Implementations
Geometric search: overview Geometric Algorithms Types of data:, lines, planes, polygons, circles,... This lecture: sets of N objects. Range searching Quadtrees, 2D trees, kd trees Intersections of geometric
More informationDynamic Bounding Volume Hierarchies. Erin Catto, Blizzard Entertainment
Dynamic Bounding Volume Hierarchies Erin Catto, Blizzard Entertainment 1 This is one of my favorite Overwatch maps: BlizzardWorld. This is the spawn area inside the Hearthstone Tavern. 2 All the objects
More informationAccelerated Raytracing
Accelerated Raytracing Why is Acceleration Important? Vanilla ray tracing is really slow! mxm pixels, kxk supersampling, n primitives, average ray path length of d, l lights, 2 recursive ray casts per
More informationAdvanced Ray Tracing
Advanced Ray Tracing Thanks to Fredo Durand and Barb Cutler The Ray Tree Ni surface normal Ri reflected ray Li shadow ray Ti transmitted (refracted) ray 51 MIT EECS 6.837, Cutler and Durand 1 Ray Tree
More informationGPU ACCELERATED SELF-JOIN FOR THE DISTANCE SIMILARITY METRIC
GPU ACCELERATED SELF-JOIN FOR THE DISTANCE SIMILARITY METRIC MIKE GOWANLOCK NORTHERN ARIZONA UNIVERSITY SCHOOL OF INFORMATICS, COMPUTING & CYBER SYSTEMS BEN KARSIN UNIVERSITY OF HAWAII AT MANOA DEPARTMENT
More informationRobot Mapping. SLAM Front-Ends. Cyrill Stachniss. Partial image courtesy: Edwin Olson 1
Robot Mapping SLAM Front-Ends Cyrill Stachniss Partial image courtesy: Edwin Olson 1 Graph-Based SLAM Constraints connect the nodes through odometry and observations Robot pose Constraint 2 Graph-Based
More informationModeling Transformations
Modeling Transformations Michael Kazhdan (601.457/657) HB Ch. 5 FvDFH Ch. 5 Overview Ra-Tracing so far Modeling transformations Ra Tracing Image RaTrace(Camera camera, Scene scene, int width, int heigh,
More informationHigh Quality Real Time Image Processing Framework on Mobile Platforms using Tegra K1. Eyal Hirsch
High Quality Real Time Image Processing Framework on Mobile Platforms using Tegra K1 Eyal Hirsch Established in 2009 and headquartered in Israel SagivTech Snapshot Core domain expertise: GPU Computing
More information