Capture of Arm-Muscle deformations using a Depth Camera

Transcription

1 Capture of Arm-Muscle deformations using a Depth Camera November 7th, 2013 Nadia Robertini 1, Thomas Neumann 2, Kiran Varanasi 3, Christian Theobalt 4 1 University of Saarland, 2 HTW Dresden, 3 Technicolor Rennes, 4 Max Planck Institut Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 1

2 Introduction

3 Introduction Skin details [Teran et al. 2005]

4 Introduction Skin details [Neumann et al. 2013] [Hasler et al. 2009] [Park & Hodgins 2006] [Teran et al. 2005]

5 Introduction [Robertini et al. 2013] Skin details [Neumann et al. 2013] [Hasler et al. 2009] [Park & Hodgins 2006] [Teran et al. 2005]

6 Method Outline Surface Reconstruction

7 Method Outline Surface Reconstruction Kinect Artifacts Shadow Flying Pixels Noise

8 Method Outline Depth-Map Pre-processing Surface Reconstruction

9 Depth-Map Pre-processing Original Segmented Thresholding

10 Depth-Map Pre-processing Original Segmented Thresholding Original (side-view) Filtered (side-view) Flying-pixels removal Median filtering Nadia Robertini, Capture of Arm-Muscle Deformations Gaussian using smoothing a Depth-Camera. #

11 Method Outline Depth-Map Pre-processing Surface Reconstruction Kinect Resolution Quantization Resolution One-side only 640 pixel 480 pixel

12 Method Outline Depth-Map Pre-processing Surface Fitting Template Mesh

13 Surface Fitting Rigid Alignment Find Corresp. Filter Corresp. As-Rigid-As-Possible Deformation [Sorkine & Alexa 2007]

14 Surface Fitting Rigid Alignment Find Corresp. Filter Corresp. As-Rigid-As-Possible Deformation Depth-points Base Mesh

15 Surface Fitting Rigid Alignment Find Corresp. Filter Corresp. As-Rigid-As-Possible Deformation Depth-points Base Mesh Corresp.

16 Surface Fitting Rigid Alignment Find Corresp. Filter Corresp. As-Rigid-As-Possible Deformation Depth-points Base Mesh Chosen Corresp. Corresp.

17 Surface Fitting Rigid Alignment Find Corresp. Filter Corresp. As-Rigid-As-Possible Deformation [Sorkine & Alexa 2007] Depth-points Base Mesh Final Mesh Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. #

18 Surface Fitting Rigid Alignment Find Corresp. Filter Corresp. As-Rigid-As-Possible Deformation Depth-Map Surface Fitting

19 Method Outline Depth-Map Pre-processing Surface Fitting Template quality Template Mesh

20 Method Outline Depth-Map Pre-processing Surface Fitting Neumann et al Dataset Hasler et al Dataset

21 Method Outline Depth-Map Pre-processing Surface Fitting Neumann et al Dataset Statistical Deformation Model Hasler et al Dataset

22 Statistical Deformation Model Physique learning Length learning Pose learning Mesh generation Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 22

23 Statistical Deformation Model Physique learning Length learning Pose learning Mesh generation S s i Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 23

24 Statistical Deformation Model Physique learning Length learning Pose learning Mesh generation S M d i s i M s i Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 24

25 Statistical Deformation Model Physique learning Length learning Pose learning Mesh generation L M d i l i M l i Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 25

26 Statistical Deformation Model Physique learning Length learning Pose learning Mesh generation P Reverse Skinning M d i p i M p i Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 26

27 Statistical Deformation Model Physique learning Length learning Pose learning Mesh generation M Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 27

28 Statistical Deformation Model Physique learning Length learning Pose learning Mesh generation M M dl Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 28

29 Statistical Deformation Model Physique learning Length learning Pose learning Mesh generation d l d d M M M l s Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 29

30 Statistical Deformation Model Physique learning Length learning Pose learning Mesh generation d l M M M dl d s M d l d s d p Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 30

31 Statistical Deformation Model Physique learning Length learning Pose learning Mesh generation d l M M M dl d s M d l d s d p d Skinning( M ) l d s d p Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 31

32 Method Outline Depth-Map Pre-processing Surface Fitting Neumann et al Dataset Statistical Deformation Model Hasler et al Dataset

33 Method Outline Depth-Map Pre-processing Surface Fitting Model-Based Filtering Neumann et al Dataset Statistical Deformation Model Hasler et al Dataset

34 Model-Based Filtering Surface Fitting

35 Model-Based Filtering Surface Fitting

36 Model-Based Filtering Surface Fitting Model-Based Filtering

37 Model-Based Filtering Surface Fitting Model-Based Filtering

38 Model-Based Filtering Surface Fitting Model-Based Filtering

39 Method Outline Depth-Map Pre-processing Surface Fitting Model-Based Filtering Neumann et al Dataset Statistical Deformation Model Hasler et al Dataset

40 Method Outline Depth-Map Pre-processing Surface Fitting Model-Based Filtering Refinement Neumann et al Dataset Statistical Deformation Model Hasler et al Dataset

41 Refinement Rigid Alignment Find Corresp. Filter Corresp. Deform (ARAP)

42 Refinement Rigid Alignment Find Corresp. Filter Corresp. Deform (ARAP) Vertex Depth-point Corresp. time

43 Refinement Rigid Alignment Find Corresp. Filter Corresp. Deform (ARAP) No Temporal Filtering Temporal Filtering

44 Refinement Rigid Alignment Find Corresp. Filter Corresp. Deform (ARAP) Model-Based Mesh Refined Mesh

45 Method Outline Depth-Map Pre-processing Surface Fitting Model-Based Filtering Refinement Neumann et al Dataset Statistical Deformation Model Hasler et al Dataset

46 Method Outline Depth-Map Pre-processing Surface Fitting Model-Based Filtering Refinement Neumann et al Dataset Statistical Deformation Model Hasler et al Dataset

47 Results

48 Results Muscular Arm Depth-Map Final Result

49 Results Muscular Arm Front Back

50 Results Muscular Arm

51 Results Skinny Arm Depth-Map Final Result

52 Results Skinny Arm Front Back

53 Results Skinny Arm

54 Results Flabby Arm Depth-Map Final Result

55 Results Flabby Arm Front Back

56 Results Flabby Arm

57 Limitations

58 Rotation Self-Occlusion Limitations Depth-Map Final Result

59 Conclusions Capture fine-scale arm-muscle deformations using the Kinect sensor Distortions-free Accurate Muscle-bulges Easy to set-up Fast acquisition Affordable

60 Thank you! Contact: Nadia Robertini Thomas Neumann Kiran Varanasi Christian Theobalt Nadia Robertini, Capture of Arm-Muscle Deformations using a Depth-Camera. 60