Robust Articulated ICP for Real-Time Hand Tracking
|
|
- Damon Green
- 6 years ago
- Views:
Transcription
1 Robust Articulated-ICP for Real-Time Hand Tracking Andrea Tagliasacchi* Sofien Bouaziz Matthias Schröder* Mario Botsch Anastasia Tkach Mark Pauly * equal contribution 1/36
2 Real-Time Tracking Setup Data from (single) RGBD Sensors PrimeSense (Carmine) SoftKinetic Intel RealSense low SNR along in depth (z-axis) completely discards small portions of geometry 2
3 Motivation? Augmented Reality Intel Perceptual SDK Oculus Research (VR) MagicLeap (AR) Microsoft HoloLens - PR Video (hololens.com) HoloLens (AR) 3
4 Previous Work Appearance-based (guess solely based on current frame) Model-based (registers model of previous frame) 4
5 Previous Work Appearance vision Model geometry [Keskin ICCV 12] [Qian CVPR 14] [Oikono. CVPR 14] [Tompson SIG 14] [Melax I3D 13] [Tang CVPR 14] [Sridhar 3DV 14] [Schroder ICRA 14] 5
6 Contributions combined 2D/3D registration (within ICP) occlusion-aware correspondences (ICP) regularization with statistical pose-space prior extensible and unified real-time solver (>60fps) revamping ICP for articulated tracking 6
7 ICP: Iterative Closest Point Step 1: optimizing correspondences Step 2: optimizing transformations update correspondences update transformation update correspondences update transformation for more details please refer to Sparse-ICP [Bouaziz, Tagliasacchi, Pauly SGP 13] 7
8 Robust Articulated ICP [Wei et al. SIGA 12] our tracking process successfully tracks the entire motion sequence while ICP fails to track most of frames. This is because ICP is often sensitive to initial poses and prone to local minimum, particularly involving tracking high-dimensional human body poses from noisy depth data. [Zhang et al. SIGA 14] The accompanying video clearly shows that our tracking process is much more robust than the ICP algorithm [ ] our tracking process successfully tracks the entire motion sequence while ICP fails to track most of frames. [Qian et al. CVPR 14] It uses alternate and gradient based optimization, converges fast, and is suitable for realtime applications. However, it can be easily trapped in poor local optima and cannot handle non-rigid objects well. Yet, it is still insufficient for high-dimensional articulated hands, especially under free viewpoints. 8
9 System Overview color image silhouette posed model lost tracking? no yes converged? no yes reinitialize 2D correspondences linear 3D correspondences data fitting solve temporal depth image distance trans. point cloud bounds pose space collision min E points + E silh. +E wrist {z } Fitting terms + E pose + E kin. + E temporal {z } Prior terms 9
10 Preprocessing S s -sensorsilhouette color to identify the Region-of-Interest demo: assumption on picking +y for PCA but all this could be learned!! [Tompson et al. TOG 14] 10
11 Data Fitting Energies color image silhouette posed model lost tracking? no yes converged? no yes reinitialize 2D correspondences linear 3D correspondences data fitting solve temporal depth image distance trans. point cloud bounds pose space collision min E points + E silh. +E wrist {z } Fitting terms + E pose + E kin. + E temporal {z } Prior terms 11
12 3D Registration (w/ occlusions) X s -sensorpointcloud E points =! 1 X x2x s kx M (x, )k 1 2 x M ICP (case #1) ICP (case #2) our method (case #2) M -cylinderhandmodel 3D Registration 12
13 Lesson #1: insufficient to render X s -sensorpointcloud Correspondences of [Wei et al. SIGA 12] (renders the hand model into a point cloud) c 1 c 2 c 1 c 2 Correspondences of [Our Method] (computes correspondences in close form) Ground Truth Motion (finger 2 comes out of occlusion) c 1 c 2 M -cylinderhandmodel 3D Registration 13
14 2D/3D Registration X s -sensorpointcloud M -cylinderhandmodel 3D Registration 14
15 2D/3D Registration X s -sensorpointcloud S s -sensorsilhouette M -cylinderhandmodel 3D Registration S r -renderedsilhouette 2D Registration 15
16 2D Registration S s -sensorsilhouette E silhouette =! 2 X p2s r kp Ss (p, )k 2 2 x Ss S r -renderedsilhouette 2D Registration 16
17 Model Prior Energies color image silhouette posed model lost tracking? no yes converged? no yes reinitialize 2D correspondences linear 3D correspondences data fitting solve temporal depth image distance trans. point cloud bounds pose space collision min E points + E silh. +E wrist {z } Fitting terms + E pose + E kin. + E temporal {z } Prior terms 17
18 Joint Bounds Energy 18
19 Collision Energy 19
20 Temporal Coherence Energy 20
21 Statistical Pose Prior encodes correlation across joints 21
22 Statistical Pose Prior Recorded by VICON tracking system [Schroder ICRA 14] (they are accurate for the person that have been recorded for) 22
23 Statistical Pose Prior (Soft) E pose =! 4 k (µ + P )k 2 2 +! 5 k k 2 2 we optimize the current pose So that it is similar to a reconstructed pose from the low dimensional subspace but when DOF are unconstrained we would like to restore the neutral (i.e. mean) pose. µ 23
24 Statistical Pose Prior (Hard) E pose =! 4 k (µ + P )k 2 2 +! 5 k k 2 2! 4 = 1 only optimize in the subspace (i.e. variable replacement) even with a high number of bases the animation remains stiff!!! 24
25 CPU/GPU Optimization min X i Ē silh. =! 2 X kf i + J i k 2 2 = X i J T i J i 1 X i J T i f i p2s r (n T (J persp (x)j skel (x) +(p Ss (p, ))) 2 S r 20k!!!! J silh 20k 26 J T silhj silh =
26 Results and Limitations 26
27 Motion Transfer to Rig 27
28 Tracking with Fast Motion 28
29 Tracking of Interacting Hands 29
30 Limitation: Calibration 30
31 Limitations: Fist Rotation 31
32 State of the Art - Evaluations Dexter-1 Dataset (MPI) mm [Shroder ICRA 14] Subspace ICP [Tang et al. 2014] [Sridhar et al. 2013] [Sridhar et al. 2014] [ours] [ours] + re-init. [Sridhar ICCV 13] EM Tracker [Tang CVPR 14] Forest Classifiers adbadd flexex1 pinch count tigergrasp wave random [Melax 14] Intel Perceptual SDK (re-initialization helps because Dexter-1 is a low frame-rate dataset only 30Hz) [Tompson TOG 14] ConvNets [Qian CVPR 14] ICP/PSO Hybrid Qualitative Quantitative 32
33 Qualitative Comparisons Convex Dynamics (Intel SDK) Subspace ICP 33
34 Conclusions combined 2D/3D registration (within ICP) occlusion-aware correspondences (ICP) regularization with statistical pose-space prior extensible and unified real-time solver (>60fps) fully open source! 34
35 Shameless Advertisement Who? Prof. Andrea Tagliasacchi and Brian Wyvill What? MSc (PhD) Where? Who pays? Language? English 35
36 Thank you!!! Live demo at SGP 15!!! Don t be shy!! Sofien Matthias Mark Anastasia 36
37 Extra Slides 37
38 Statistical Prior: Side Effects E pose =! 4 k (µ + P )k 2 2 +! 5 k k as this prior correlates joint angles the convergence speed increases by about 3x 38
39 Reinitialization from Failure Determine tracking failure [Melax 13] Detection by ~[Qian et al. CVPR 14] 1 = X x2x s kx M (x, )k 2, 2 = S r \ S s S r [ S s xy-fingers Logistic Regressor (optimize alpha s.t.): 2 1+e T [ 1, 2 ]! tracking ok! z-fingers Exploration of corresp. energy 39
40 Particle Swarm 40
Accurate 3D Face and Body Modeling from a Single Fixed Kinect
Accurate 3D Face and Body Modeling from a Single Fixed Kinect Ruizhe Wang*, Matthias Hernandez*, Jongmoo Choi, Gérard Medioni Computer Vision Lab, IRIS University of Southern California Abstract In this
More informationIntrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting R. Maier 1,2, K. Kim 1, D. Cremers 2, J. Kautz 1, M. Nießner 2,3 Fusion Ours 1
More informationDeep Learning for Virtual Shopping. Dr. Jürgen Sturm Group Leader RGB-D
Deep Learning for Virtual Shopping Dr. Jürgen Sturm Group Leader RGB-D metaio GmbH Augmented Reality with the Metaio SDK: IKEA Catalogue App Metaio: Augmented Reality Metaio SDK for ios, Android and Windows
More informationDynamic Geometry Processing
Dynamic Geometry Processing EG 2012 Tutorial Will Chang, Hao Li, Niloy Mitra, Mark Pauly, Michael Wand Tutorial: Dynamic Geometry Processing 1 Articulated Global Registration Introduction and Overview
More informationDynamic 2D/3D Registration for the Kinect
Dynamic 2D/3D Registration for the Kinect Sofien Bouaziz Mark Pauly École Polytechnique Fédérale de Lausanne Abstract Image and geometry registration algorithms are an essential component of many computer
More informationA novel approach to motion tracking with wearable sensors based on Probabilistic Graphical Models
A novel approach to motion tracking with wearable sensors based on Probabilistic Graphical Models Emanuele Ruffaldi Lorenzo Peppoloni Alessandro Filippeschi Carlo Alberto Avizzano 2014 IEEE International
More informationBack To RGB: Deep Articulated Hand Pose Estimation From a Single Camera Image
Back To RGB: Deep Articulated Hand Pose Estimation From a Single Camera Image Wan-Duo Kurt Ma, J.P. Lewis, Marcus Frean and David Balduzzi School of Engineering and Computer Science, Victoria University
More information3D Computer Vision. Structured Light II. Prof. Didier Stricker. Kaiserlautern University.
3D Computer Vision Structured Light II Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Introduction
More informationSingle-shot Extrinsic Calibration of a Generically Configured RGB-D Camera Rig from Scene Constraints
Single-shot Extrinsic Calibration of a Generically Configured RGB-D Camera Rig from Scene Constraints Jiaolong Yang*, Yuchao Dai^, Hongdong Li^, Henry Gardner^ and Yunde Jia* *Beijing Institute of Technology
More informationStructured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov
Structured Light II Johannes Köhler Johannes.koehler@dfki.de Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov Introduction Previous lecture: Structured Light I Active Scanning Camera/emitter
More informationLecture 19: Depth Cameras. Visual Computing Systems CMU , Fall 2013
Lecture 19: Depth Cameras Visual Computing Systems Continuing theme: computational photography Cameras capture light, then extensive processing produces the desired image Today: - Capturing scene depth
More informationChaplin, Modern Times, 1936
Chaplin, Modern Times, 1936 [A Bucket of Water and a Glass Matte: Special Effects in Modern Times; bonus feature on The Criterion Collection set] Multi-view geometry problems Structure: Given projections
More informationMultiframe Scene Flow with Piecewise Rigid Motion. Vladislav Golyanik,, Kihwan Kim, Robert Maier, Mathias Nießner, Didier Stricker and Jan Kautz
Multiframe Scene Flow with Piecewise Rigid Motion Vladislav Golyanik,, Kihwan Kim, Robert Maier, Mathias Nießner, Didier Stricker and Jan Kautz Scene Flow. 2 Scene Flow. 3 Scene Flow. Scene Flow Estimation:
More informationNonrigid Surface Modelling. and Fast Recovery. Department of Computer Science and Engineering. Committee: Prof. Leo J. Jia and Prof. K. H.
Nonrigid Surface Modelling and Fast Recovery Zhu Jianke Supervisor: Prof. Michael R. Lyu Committee: Prof. Leo J. Jia and Prof. K. H. Wong Department of Computer Science and Engineering May 11, 2007 1 2
More informationRegistration of Dynamic Range Images
Registration of Dynamic Range Images Tan-Chi Ho 1,2 Jung-Hong Chuang 1 Wen-Wei Lin 2 Song-Sun Lin 2 1 Department of Computer Science National Chiao-Tung University 2 Department of Applied Mathematics National
More informationColored Point Cloud Registration Revisited Supplementary Material
Colored Point Cloud Registration Revisited Supplementary Material Jaesik Park Qian-Yi Zhou Vladlen Koltun Intel Labs A. RGB-D Image Alignment Section introduced a joint photometric and geometric objective
More informationReal-time Image-based Reconstruction of Pipes Using Omnidirectional Cameras
Real-time Image-based Reconstruction of Pipes Using Omnidirectional Cameras Dipl. Inf. Sandro Esquivel Prof. Dr.-Ing. Reinhard Koch Multimedia Information Processing Christian-Albrechts-University of Kiel
More informationDynamic Time Warping for Binocular Hand Tracking and Reconstruction
Dynamic Time Warping for Binocular Hand Tracking and Reconstruction Javier Romero, Danica Kragic Ville Kyrki Antonis Argyros CAS-CVAP-CSC Dept. of Information Technology Institute of Computer Science KTH,
More informationUser-assisted Segmentation and 3D Motion Tracking. Michael Fleder Sudeep Pillai Jeremy Scott
User-assisted Segmentation and 3D Motion Tracking Michael Fleder Sudeep Pillai Jeremy Scott 3D Object Tracking Virtual reality and animation Imitation in robotics Autonomous driving Augmented reality Motivation
More informationLearning 6D Object Pose Estimation and Tracking
Learning 6D Object Pose Estimation and Tracking Carsten Rother presented by: Alexander Krull 21/12/2015 6D Pose Estimation Input: RGBD-image Known 3D model Output: 6D rigid body transform of object 21/12/2015
More informationInternational Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XXXIV-5/W10
BUNDLE ADJUSTMENT FOR MARKERLESS BODY TRACKING IN MONOCULAR VIDEO SEQUENCES Ali Shahrokni, Vincent Lepetit, Pascal Fua Computer Vision Lab, Swiss Federal Institute of Technology (EPFL) ali.shahrokni,vincent.lepetit,pascal.fua@epfl.ch
More informationMonocular Tracking and Reconstruction in Non-Rigid Environments
Monocular Tracking and Reconstruction in Non-Rigid Environments Kick-Off Presentation, M.Sc. Thesis Supervisors: Federico Tombari, Ph.D; Benjamin Busam, M.Sc. Patrick Ruhkamp 13.01.2017 Introduction Motivation:
More informationTEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA
TEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA Tomoki Hayashi 1, Francois de Sorbier 1 and Hideo Saito 1 1 Graduate School of Science and Technology, Keio University, 3-14-1 Hiyoshi,
More informationSurfNet: Generating 3D shape surfaces using deep residual networks-supplementary Material
SurfNet: Generating 3D shape surfaces using deep residual networks-supplementary Material Ayan Sinha MIT Asim Unmesh IIT Kanpur Qixing Huang UT Austin Karthik Ramani Purdue sinhayan@mit.edu a.unmesh@gmail.com
More informationAccurate, Robust, and Flexible Real-time Hand Tracking
Accurate, Robust, and Flexible Real-time Hand Tracking Toby Sharp Cem Keskin Duncan Robertson Jonathan Taylor Jamie Shotton David Kim Christoph Rhemann Ido Leichter Alon Vinnikov Yichen Wei Daniel Freedman
More informationarxiv: v3 [cs.cv] 10 Jan 2018
HAND SEGMENTATION FOR HAND-OBJECT INTERACTION FROM DEPTH MAP Byeongkeun Kang Kar-Han Tan Nan Jiang Hung-Shuo Tai Daniel Tretter Truong Nguyen Department of Electrical and Computer Engineering, UC San Diego,
More informationModeling 3D Human Poses from Uncalibrated Monocular Images
Modeling 3D Human Poses from Uncalibrated Monocular Images Xiaolin K. Wei Texas A&M University xwei@cse.tamu.edu Jinxiang Chai Texas A&M University jchai@cse.tamu.edu Abstract This paper introduces an
More informationDense 3D Reconstruction from Autonomous Quadrocopters
Dense 3D Reconstruction from Autonomous Quadrocopters Computer Science & Mathematics TU Munich Martin Oswald, Jakob Engel, Christian Kerl, Frank Steinbrücker, Jan Stühmer & Jürgen Sturm Autonomous Quadrocopters
More information3D Reconstruction with Tango. Ivan Dryanovski, Google Inc.
3D Reconstruction with Tango Ivan Dryanovski, Google Inc. Contents Problem statement and motivation The Tango SDK 3D reconstruction - data structures & algorithms Applications Developer tools Problem formulation
More informationHigh-Fidelity Augmented Reality Interactions Hrvoje Benko Researcher, MSR Redmond
High-Fidelity Augmented Reality Interactions Hrvoje Benko Researcher, MSR Redmond New generation of interfaces Instead of interacting through indirect input devices (mice and keyboard), the user is interacting
More informationTEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA
TEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA Tomoki Hayashi, Francois de Sorbier and Hideo Saito Graduate School of Science and Technology, Keio University, 3-14-1 Hiyoshi, Kohoku-ku,
More informationHigh-Fidelity Facial and Speech Animation for VR HMDs
High-Fidelity Facial and Speech Animation for VR HMDs Institute of Computer Graphics and Algorithms Vienna University of Technology Forecast facial recognition with Head-Mounted Display (HMD) two small
More information4422 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 9, SEPTEMBER 2018
4422 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 9, SEPTEMBER 2018 Robust 3D Hand Pose Estimation From Single Depth Images Using Multi-View CNNs Liuhao Ge, Hui Liang, Member, IEEE, Junsong Yuan,
More informationAugmenting Reality, Naturally:
Augmenting Reality, Naturally: Scene Modelling, Recognition and Tracking with Invariant Image Features by Iryna Gordon in collaboration with David G. Lowe Laboratory for Computational Intelligence Department
More informationPlanning, Execution and Learning Application: Examples of Planning in Perception
15-887 Planning, Execution and Learning Application: Examples of Planning in Perception Maxim Likhachev Robotics Institute Carnegie Mellon University Two Examples Graph search for perception Planning for
More informationOverview. Related Work Tensor Voting in 2-D Tensor Voting in 3-D Tensor Voting in N-D Application to Vision Problems Stereo Visual Motion
Overview Related Work Tensor Voting in 2-D Tensor Voting in 3-D Tensor Voting in N-D Application to Vision Problems Stereo Visual Motion Binary-Space-Partitioned Images 3-D Surface Extraction from Medical
More informationTri-modal Human Body Segmentation
Tri-modal Human Body Segmentation Master of Science Thesis Cristina Palmero Cantariño Advisor: Sergio Escalera Guerrero February 6, 2014 Outline 1 Introduction 2 Tri-modal dataset 3 Proposed baseline 4
More information3D object recognition used by team robotto
3D object recognition used by team robotto Workshop Juliane Hoebel February 1, 2016 Faculty of Computer Science, Otto-von-Guericke University Magdeburg Content 1. Introduction 2. Depth sensor 3. 3D object
More informationSegmentation and Tracking of Partial Planar Templates
Segmentation and Tracking of Partial Planar Templates Abdelsalam Masoud William Hoff Colorado School of Mines Colorado School of Mines Golden, CO 800 Golden, CO 800 amasoud@mines.edu whoff@mines.edu Abstract
More informationACQUIRING 3D models of deforming objects in real-life is
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 1 Robust Non-rigid Motion Tracking and Surface Reconstruction Using L 0 Regularization Kaiwen Guo, Feng Xu, Yangang Wang, Yebin Liu, Member, IEEE
More informationPerceptual Grouping from Motion Cues Using Tensor Voting
Perceptual Grouping from Motion Cues Using Tensor Voting 1. Research Team Project Leader: Graduate Students: Prof. Gérard Medioni, Computer Science Mircea Nicolescu, Changki Min 2. Statement of Project
More informationCompressive Sensing: Opportunities and Perils for Computer Vision
Compressive Sensing: Opportunities and Perils for Computer Vision Rama Chellappa And Volkan Cevher (Rice) Joint work with Aswin C. Sankaranarayanan Dikpal Reddy Dr. Ashok Veeraraghavan (MERL) Prof. Rich
More information3D Human Motion Analysis and Manifolds
D E P A R T M E N T O F C O M P U T E R S C I E N C E U N I V E R S I T Y O F C O P E N H A G E N 3D Human Motion Analysis and Manifolds Kim Steenstrup Pedersen DIKU Image group and E-Science center Motivation
More informationOutline. 1 Why we re interested in Real-Time tracking and mapping. 3 Kinect Fusion System Overview. 4 Real-time Surface Mapping
Outline CSE 576 KinectFusion: Real-Time Dense Surface Mapping and Tracking PhD. work from Imperial College, London Microsoft Research, Cambridge May 6, 2013 1 Why we re interested in Real-Time tracking
More information3D Face and Hand Tracking for American Sign Language Recognition
3D Face and Hand Tracking for American Sign Language Recognition NSF-ITR (2004-2008) D. Metaxas, A. Elgammal, V. Pavlovic (Rutgers Univ.) C. Neidle (Boston Univ.) C. Vogler (Gallaudet) The need for automated
More informationVirtual Reality in Assembly Simulation
Virtual Reality in Assembly Simulation Collision Detection, Simulation Algorithms, and Interaction Techniques Contents 1. Motivation 2. Simulation and description of virtual environments 3. Interaction
More informationCRF Based Point Cloud Segmentation Jonathan Nation
CRF Based Point Cloud Segmentation Jonathan Nation jsnation@stanford.edu 1. INTRODUCTION The goal of the project is to use the recently proposed fully connected conditional random field (CRF) model to
More informationRobust Model-Free Tracking of Non-Rigid Shape. Abstract
Robust Model-Free Tracking of Non-Rigid Shape Lorenzo Torresani Stanford University ltorresa@cs.stanford.edu Christoph Bregler New York University chris.bregler@nyu.edu New York University CS TR2003-840
More information3D Reconstruction of Dynamic Textures with Crowd Sourced Data. Dinghuang Ji, Enrique Dunn and Jan-Michael Frahm
3D Reconstruction of Dynamic Textures with Crowd Sourced Data Dinghuang Ji, Enrique Dunn and Jan-Michael Frahm 1 Background Large scale scene reconstruction Internet imagery 3D point cloud Dense geometry
More informationECE 6554:Advanced Computer Vision Pose Estimation
ECE 6554:Advanced Computer Vision Pose Estimation Sujay Yadawadkar, Virginia Tech, Agenda: Pose Estimation: Part Based Models for Pose Estimation Pose Estimation with Convolutional Neural Networks (Deep
More informationRobotics Programming Laboratory
Chair of Software Engineering Robotics Programming Laboratory Bertrand Meyer Jiwon Shin Lecture 8: Robot Perception Perception http://pascallin.ecs.soton.ac.uk/challenges/voc/databases.html#caltech car
More information3D Perception. CS 4495 Computer Vision K. Hawkins. CS 4495 Computer Vision. 3D Perception. Kelsey Hawkins Robotics
CS 4495 Computer Vision Kelsey Hawkins Robotics Motivation What do animals, people, and robots want to do with vision? Detect and recognize objects/landmarks Find location of objects with respect to themselves
More informationComputer Vision II Lecture 14
Computer Vision II Lecture 14 Articulated Tracking I 08.07.2014 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Outline of This Lecture Single-Object Tracking Bayesian
More informationPersonal Navigation and Indoor Mapping: Performance Characterization of Kinect Sensor-based Trajectory Recovery
Personal Navigation and Indoor Mapping: Performance Characterization of Kinect Sensor-based Trajectory Recovery 1 Charles TOTH, 1 Dorota BRZEZINSKA, USA 2 Allison KEALY, Australia, 3 Guenther RETSCHER,
More information3D Photography: Active Ranging, Structured Light, ICP
3D Photography: Active Ranging, Structured Light, ICP Kalin Kolev, Marc Pollefeys Spring 2013 http://cvg.ethz.ch/teaching/2013spring/3dphoto/ Schedule (tentative) Feb 18 Feb 25 Mar 4 Mar 11 Mar 18 Mar
More informationFast and Robust Hand Tracking Using Detection-Guided Optimization
Fast and Robust Hand Tracking Using Detection-Guided Optimization Srinath Sridhar 1, Franziska Mueller 1,2, Antti Oulasvirta 3, Christian Theobalt 1 1 Max Planck Institute for Informatics, 2 Saarland University,
More informationTask analysis based on observing hands and objects by vision
Task analysis based on observing hands and objects by vision Yoshihiro SATO Keni Bernardin Hiroshi KIMURA Katsushi IKEUCHI Univ. of Electro-Communications Univ. of Karlsruhe Univ. of Tokyo Abstract In
More informationThis document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore.
This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore. Title Egocentric hand pose estimation and distance recovery in a single RGB image Author(s) Liang, Hui; Yuan,
More informationLearning Articulated Skeletons From Motion
Learning Articulated Skeletons From Motion Danny Tarlow University of Toronto, Machine Learning with David Ross and Richard Zemel (and Brendan Frey) August 6, 2007 Point Light Displays It's easy for humans
More informationModel-Based Reinforcement of Kinect Depth Data for Human Motion Capture Applications
Sensors 2013, 13, 8835-8855; doi:10.3390/s130708835 OPEN ACCESS sensors ISSN 1424-8220 www.mdpi.com/journal/sensors Article Model-Based Reinforcement of Kinect Depth Data for Human Motion Capture Applications
More informationWhat have we leaned so far?
What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging What have we learned so far? Image Filtering Image Warping Camera Projection Model Project 2: Panoramic
More informationDynamic Shape Tracking via Region Matching
Dynamic Shape Tracking via Region Matching Ganesh Sundaramoorthi Asst. Professor of EE and AMCS KAUST (Joint work with Yanchao Yang) The Problem: Shape Tracking Given: exact object segmentation in frame1
More informationDense 3D Modelling and Monocular Reconstruction of Deformable Objects
Dense 3D Modelling and Monocular Reconstruction of Deformable Objects Anastasios (Tassos) Roussos Lecturer in Computer Science, University of Exeter Research Associate, Imperial College London Overview
More informationLUMS Mine Detector Project
LUMS Mine Detector Project Using visual information to control a robot (Hutchinson et al. 1996). Vision may or may not be used in the feedback loop. Visual (image based) features such as points, lines
More informationShape from Selfies : Human Body Shape Estimation using CCA Regression Forests
Shape from Selfies : Human Body Shape Estimation using CCA Regression Forests Endri Dibra 1 Cengiz Öztireli1 Remo Ziegler 2 Markus Gross 1 1 Department of Computer Science, ETH Zürich 2 Vizrt {edibra,cengizo,grossm}@inf.ethz.ch
More informationSurface Registration. Gianpaolo Palma
Surface Registration Gianpaolo Palma The problem 3D scanning generates multiple range images Each contain 3D points for different parts of the model in the local coordinates of the scanner Find a rigid
More informationIterative Estimation of 3D Transformations for Object Alignment
Iterative Estimation of 3D Transformations for Object Alignment Tao Wang and Anup Basu Department of Computing Science, Univ. of Alberta, Edmonton, AB T6G 2E8, Canada Abstract. An Iterative Estimation
More informationThe Future of Natural User Interaction
The Future of Natural User Interaction NUI: a great new discipline in the making Baining Guo Microsoft Research Asia Kinect from Insiders Perspective An interview with Kinect contributors Kinect from
More informationAugmented Reality VU. Computer Vision 3D Registration (2) Prof. Vincent Lepetit
Augmented Reality VU Computer Vision 3D Registration (2) Prof. Vincent Lepetit Feature Point-Based 3D Tracking Feature Points for 3D Tracking Much less ambiguous than edges; Point-to-point reprojection
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationLocalization, Mapping and Exploration with Multiple Robots. Dr. Daisy Tang
Localization, Mapping and Exploration with Multiple Robots Dr. Daisy Tang Two Presentations A real-time algorithm for mobile robot mapping with applications to multi-robot and 3D mapping, by Thrun, Burgard
More informationarxiv: v1 [cs.cv] 28 Sep 2018
Camera Pose Estimation from Sequence of Calibrated Images arxiv:1809.11066v1 [cs.cv] 28 Sep 2018 Jacek Komorowski 1 and Przemyslaw Rokita 2 1 Maria Curie-Sklodowska University, Institute of Computer Science,
More informationGrasp Recognition using a 3D Articulated Model and Infrared Images
Grasp Recognition using a 3D Articulated Model and Infrared Images Koichi Ogawara Institute of Industrial Science, Univ. of Tokyo, Tokyo, Japan Jun Takamatsu Institute of Industrial Science, Univ. of Tokyo,
More informationRecovering 6D Object Pose and Predicting Next-Best-View in the Crowd Supplementary Material
Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd Supplementary Material Andreas Doumanoglou 1,2, Rigas Kouskouridas 1, Sotiris Malassiotis 2, Tae-Kyun Kim 1 1 Imperial College London
More informationMultiple View Geometry
Multiple View Geometry Martin Quinn with a lot of slides stolen from Steve Seitz and Jianbo Shi 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Our Goal The Plenoptic Function P(θ,φ,λ,t,V
More informationCS 775: Advanced Computer Graphics. Lecture 17 : Motion Capture
CS 775: Advanced Computer Graphics Lecture 17 : History Study of human motion Leonardo da Vinci (1452 1519) History Study of human motion Edward J. Muybridge, 1830 1904 http://en.wikipedia.org/wiki/eadweard_muybridge
More informationHuman Upper Body Pose Estimation in Static Images
1. Research Team Human Upper Body Pose Estimation in Static Images Project Leader: Graduate Students: Prof. Isaac Cohen, Computer Science Mun Wai Lee 2. Statement of Project Goals This goal of this project
More informationKinectFusion: Real-Time Dense Surface Mapping and Tracking
KinectFusion: Real-Time Dense Surface Mapping and Tracking Gabriele Bleser Thanks to Richard Newcombe for providing the ISMAR slides Overview General: scientific papers (structure, category) KinectFusion:
More informationComputer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
More informationECCV Presented by: Boris Ivanovic and Yolanda Wang CS 331B - November 16, 2016
ECCV 2016 Presented by: Boris Ivanovic and Yolanda Wang CS 331B - November 16, 2016 Fundamental Question What is a good vector representation of an object? Something that can be easily predicted from 2D
More informationDoes Dimensionality Reduction Improve the Quality of Motion Interpolation?
Does Dimensionality Reduction Improve the Quality of Motion Interpolation? Sebastian Bitzer, Stefan Klanke and Sethu Vijayakumar School of Informatics - University of Edinburgh Informatics Forum, 10 Crichton
More informationComputer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
More informationCorrespondence. CS 468 Geometry Processing Algorithms. Maks Ovsjanikov
Shape Matching & Correspondence CS 468 Geometry Processing Algorithms Maks Ovsjanikov Wednesday, October 27 th 2010 Overall Goal Given two shapes, find correspondences between them. Overall Goal Given
More informationSingle Camera Calibration
Single Camera Calibration using Partially Visible Calibration Objects Based on Random Dots Marker Tracking Algorithm *Yuji Oyamada1,2, Pascal Fallavollita2, and Nassir Navab2 1. Keio University, Japan
More informationa real convergence computer vision finally becoming useful
Introduce the field a real convergence computer vision finally becoming useful Pikachu Nintendo - http://www.freelargeimages.com/wpcontent/uploads/2014/12/pikachu_04.png HoloLens Microsoft - http://www.glassappsource.com/wpcontent/uploads/2016/01/hololens.jpg
More informationCombining PGMs and Discriminative Models for Upper Body Pose Detection
Combining PGMs and Discriminative Models for Upper Body Pose Detection Gedas Bertasius May 30, 2014 1 Introduction In this project, I utilized probabilistic graphical models together with discriminative
More informationPROBLEM FORMULATION AND RESEARCH METHODOLOGY
PROBLEM FORMULATION AND RESEARCH METHODOLOGY ON THE SOFT COMPUTING BASED APPROACHES FOR OBJECT DETECTION AND TRACKING IN VIDEOS CHAPTER 3 PROBLEM FORMULATION AND RESEARCH METHODOLOGY The foregoing chapter
More informationOn-line and Off-line 3D Reconstruction for Crisis Management Applications
On-line and Off-line 3D Reconstruction for Crisis Management Applications Geert De Cubber Royal Military Academy, Department of Mechanical Engineering (MSTA) Av. de la Renaissance 30, 1000 Brussels geert.de.cubber@rma.ac.be
More informationStructure-aware 3D Hand Pose Regression from a Single Depth Image
Structure-aware 3D Hand Pose Regression from a Single Depth Image Jameel Malik 1,2, Ahmed Elhayek 1, and Didier Stricker 1 1 Department Augmented Vision, DFKI Kaiserslautern, Germany 2 NUST-SEECS, Pakistan
More informationVirtualized Reality Using Depth Camera Point Clouds
Virtualized Reality Using Depth Camera Point Clouds Jordan Cazamias Stanford University jaycaz@stanford.edu Abhilash Sunder Raj Stanford University abhisr@stanford.edu Abstract We explored various ways
More informationA Model-based Approach to Rapid Estimation of Body Shape and Postures Using Low-Cost Depth Cameras
A Model-based Approach to Rapid Estimation of Body Shape and Postures Using Low-Cost Depth Cameras Abstract Byoung-Keon D. PARK*, Matthew P. REED University of Michigan, Transportation Research Institute,
More informationLeveraging Depth Cameras and Wearable Pressure Sensors for Full-body Kinematics and Dynamics Capture
Leveraging Depth Cameras and Wearable Pressure Sensors for Full-body Kinematics and Dynamics Capture Peizhao Zhang 1 Kristin Siu 2 Jianjie Zhang 1 C. Karen Liu 2 Jinxiang Chai 1 1 Texas 2 Georgia A&M University
More information3D Geometric Computer Vision. Martin Jagersand Univ. Alberta Edmonton, Alberta, Canada
3D Geometric Computer Vision Martin Jagersand Univ. Alberta Edmonton, Alberta, Canada Multi-view geometry Build 3D models from images Carry out manipulation tasks http://webdocs.cs.ualberta.ca/~vis/ibmr/
More informationTracking. Hao Guan( 管皓 ) School of Computer Science Fudan University
Tracking Hao Guan( 管皓 ) School of Computer Science Fudan University 2014-09-29 Multimedia Video Audio Use your eyes Video Tracking Use your ears Audio Tracking Tracking Video Tracking Definition Given
More informationDirect Plane Tracking in Stereo Images for Mobile Navigation
Direct Plane Tracking in Stereo Images for Mobile Navigation Jason Corso, Darius Burschka,Greg Hager Computational Interaction and Robotics Lab 1 Input: The Problem Stream of rectified stereo images, known
More informationRobust Human Body Shape and Pose Tracking
Robust Human Body Shape and Pose Tracking Chun-Hao Huang 1 Edmond Boyer 2 Slobodan Ilic 1 1 Technische Universität München 2 INRIA Grenoble Rhône-Alpes Marker-based motion capture (mocap.) Adventages:
More informationMotion Tracking and Event Understanding in Video Sequences
Motion Tracking and Event Understanding in Video Sequences Isaac Cohen Elaine Kang, Jinman Kang Institute for Robotics and Intelligent Systems University of Southern California Los Angeles, CA Objectives!
More informationStereo and Epipolar geometry
Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka
More informationA consumer level 3D object scanning device using Kinect for web-based C2C business
A consumer level 3D object scanning device using Kinect for web-based C2C business Geoffrey Poon, Yu Yin Yeung and Wai-Man Pang Caritas Institute of Higher Education Introduction Internet shopping is popular
More informationPose estimation using a variety of techniques
Pose estimation using a variety of techniques Keegan Go Stanford University keegango@stanford.edu Abstract Vision is an integral part robotic systems a component that is needed for robots to interact robustly
More information