Transparent, Sensation- Preserving Haptic Rendering

Transcription

1 Transparent, Sensation- Preserving Haptic Rendering Miguel A. Otaduy IEEE Virtual Reality Conference 2007 Tutorial 3: Integration of Haptics in Virtual Environments A Perception-Based Approach

2 Manipulation and Contact Lifting and grasping Object insertion/extraction Controlled force exertion Kinesthetic perception of contact during manipulation

3 Kinesthetic Feedback Molecular docking [UNC-Chapel Hill] Medical simulation [Reachin AB] Prototyping [CEA/Haption] Airplane design [Boeing]

4 Haptically Perceived Properties Mechanical properties (inertia, weight, compliance ) Geometric features Surface properties (texture, roughness ) Contact constraints

5 High Fidelity Perceptual accuracy Responsiveness Stability

6 Perception of Contact in VEs Haptic: constraints, features, texture Visual: deformation, non-penetration Auditory: noise Cross-modal interaction

7 6-DoF Haptic Rendering

8 Overview Rendering Algorithm Geometric Features (Collision Detection) Contact Forces (Collision Response) Transparency (Multirate Rendering)

9 Overview Rendering Algorithm Geometric Features (Collision Detection) Contact Forces (Collision Response) Transparency (Multirate Rendering)

10 Naïve Penalty-Based Approach x Haptic Device and Controller F Tool Environment Collision detection: penetration depth (PD) computation. Collision response: penalty-based F = k * PD.

11 Mechanical Impedance (Z) Haptic Device and Controller x F Z (Haptic Rendering) Force (and torque) as a reaction to the motion of the user

12 Mechanical Impedance (Z) Nonlinear differential operator. Stable Vs. unstable (from control theory). Depends on: Stiffness of penalty-based response. Haptic loop update rate. Geometry of the models. Number of contacts. Higher frame rate higher stiffness, but it is practically impossible to tune.

13 Virtual Coupling Nonlinear multidimensional VEs Multidimensional viscoelastic coupling [Colgate et al. 1995]

14 Virtual Coupling

15 Rendering Algorithm Haptic Device and Controller Tool (Rigid Body Simulation) Collision Detection and Response Virtual Coupling High update rate for stable and transparent rendering

16 Rendering Algorithm Haptic Device and Controller Rigid Body Simulation Collision Detection and Response Virtual Coupling 1 khz Intermediate Representation ~100 Hz

17 Rendering Algorithm Haptic Device and Controller Rigid Body Simulation Collision Detection and Response Virtual Coupling Intermediate Representation [McNeely et al. 1999, Johnson et al. 2005, Otaduy and Lin 2006, Ortega et al. 2006, and more ]

18 Sensation-Preserving Rendering Haptic Device and Controller Rigid Body Simulation Collision Detection and Response Virtual Coupling Intermediate Representation 1) Fast collision detection with perceptuallyaccurate geometry.

19 Sensation-Preserving Rendering Haptic Device and Controller Rigid Body Simulation Collision Detection and Response Virtual Coupling Intermediate Representation 2) Perceptually-accurate force models.

20 Sensation-Preserving Rendering Haptic Device and Controller Rigid Body Simulation Collision Detection and Response Virtual Coupling Intermediate Representation 3) Transparent display: high-stiffness coupling, possibly small tool mass, accurate intermediate representation, high update rate.

21 Overview Rendering Algorithm Geometric Features (Collision Detection) Contact Forces (Collision Response) Transparency (Multirate Rendering)

22 Perceptual Motivation Larger contact area Lower resolution Supported by studies on feature detection

23 Goal: Adaptive Resolution

24 Goal: Adaptive Resolution low res high res

25 Contact Levels of Detail (CLODs) Unique hierarchy with dual functionality: levels of detail (LODs) and bounding volume hierarchy (BVH). Exploit hierarchical nature of LODs and BVHs Create LOD hierarchy and BVH simultaneously Descend on BVH = Refine LODs [Otaduy and Lin 2003a, Otaduy and Lin 2003b]

26 Example of Hierarchy Input obj: 40K tris LOD 0: pieces LOD 3: 1414 pieces LOD 6: 176 pieces LOD 11: 8 pieces LOD 14: 1 piece

27 Example of Hierarchy Input obj: 40K tris LOD 0: pieces LOD 1: 5661 pieces LOD 2: 2830 pieces LOD 4: 707 pieces LOD 7: 88 pieces

28 Collision Detection: BVHs no collision collision a b a 1 a 2 b 1 b 2 contact Bounding Volume Test Tree (BVTT) BVHs of objects A and B

29 Collision Detection: CLODs no collision collision no refinement a b a 1 a 2 b 1 b 2 Selective Refinement BVHs of objects A and B

30 Collision Detection: CLODs no collision collision no refinement a b a 1 a 2 b 1 b 2 r i r j Adaptive resolution (r) BVHs of objects A and B

31 Error Metrics Refine if error(a, b) = s* ab > ε Error: weighted surface deviation s * ab = max s r a 2 a D, s r b 2 b s: surface deviation r: resolution D: estimated contact area ε: 3% of object radius (user studies)

32 Performance: ~300 Hz (in 2003) Error: 2.5 % - 3 % object radii club: 104,888 tris ball: 177,876 tris upper jaw: lower jaw: 47,339 tris 40,180 tris joint: tris

33 Limitations Lack of containment virtual prototyping applications do not allow interpenetration Limited simplification aggressiveness due to topological constraints Static LODs and popping effects

34 Generalization CLODs data structure independent of BV OBBs [Yoon et al. 2004] Application to rigid body simulation Extension to other collision detection algs. Voxel sampling Normal cone hierarchies Continuous collision detection

35 Limitations (2) Driving observation: small features cannot be perceived if the contact area is large Does not hold for: Highly correlated features, Tangential motion.

36 Textured Objects

37 Haptic Texture Mapping Coarse geometric representations Haptic textures [Otaduy et al. 2004]

38 Penetration Depth: Definition δ = Minimum translational distance to separate two intersecting objects δ

39 Two-stage PD Computation

40 Step 1: Approximate PD

41 Step 1: Approximate PD

42 Step 2: Refined PD

43 Pass 1: Render Geometry

44 Pass 1: Texture Mapping

45 Discrete Height Fields

46 Pass 2: Subtract Height Fields

47 Find Maximum

48 Complex Objects (>500Ktris) Performance: Hz (in 2004)

49 Extension: Deformable Textures [Galoppo et al. 2006]

50 Extension: Deformable Textures

51 Overview Rendering Algorithm Geometric Features (Collision Detection) Contact Forces (Collision Response) Transparency (Multirate Rendering)

52 Force Model Penetration depth: Applicable to arbitrary object-object interaction Also used in previous 3-DoF rendering methods Penalty-based potential field: U = 1 kδ 2 2 [Otaduy and Lin 2004]

53 ( ) ( ) = = = n n v n u n n n n n n n v u n v u v u k k U T T T F F F θ δ θ δ θ δ δ δ δ δ δ 1 Force Model Determine penetration direction n Force and Torque = Gradient of energy:

54 Roughness under Translation z y x Position (mm.) x y z Forces (N) F x F y F z Simulation frames

55 Roughness under Rotation 6 Motion along n (in mm.) 4 n Rotation around n (in deg.) Simulation frames

56 Validation Studies on perception of textures through a rigid probe by Klatzky and Lederman [2002] Analyze effects of probe diameter, applied force and exploratory speed Inspiration for our force model

57 Effect of Probe Diameter Studies by Klatzky and Lederman Simulation results Maximum Acceleration mm 2mm 3mm Texture Spacing (mm)

58 Effect of Applied Force Studies by Klatzky and Lederman Simulation results Maximum Acceleration N 0.58N 0.87N Texture Spacing (mm)

59 Limitations Surfaces must be described as height fields in the contact region Possible sampling-related aliasing Limited stability with high PD gradient

60 Overview Rendering Algorithm Geometric Features (Collision Detection) Contact Forces (Collision Response) Transparency (Multirate Rendering)

61 Rendering Algorithm Haptic Device and Controller Rigid Body Simulation Collision Detection and Response Virtual Coupling Intermediate Representation

62 Implicit Integration [Otaduy and Lin 2006] Haptic Device and Controller Implicit Integration of Rigid Body Sim. Collision Detection and Response Virtual Coupling Intermediate Representation Small tool mass More transparent rendering. High contact stiffness Contact is perceived (visually) as stiff. Displayed stiffness still limited by frame rate.

63 Linearized Contact Model Haptic Device and Controller Implicit Integration of Rigid Body Sim. Collision Detection and Response Virtual Coupling 1 khz Linearized Contact Model ~100 Hz Low frequency update of forces and their derivatives. High frequency evaluation for tool simulation.

64 Linearized Contact Model Penalty-based response for rigid tool rigid environment [Otaduy and Lin 2006]. Constraint-based response for rigid tool deformable environment [Otaduy and Gross 2007].

65 Rigid Body Simulation Motion equations Implicit velocity update requires Jacobians and

66 Penalty-Based Response Linearized model:

67 Linearized Penalty-Based Response with CLODs

68 Deformable Envt. Simulation Motion equations Implicit velocity update

69 Constraint-Based Response Velocity constraints Add contact constraint forces: Linearized model:

70 Linearized Constraint-Based Response with Deformable Envt.

71 Summary Rendering Algorithm - Virtual Coupling - Intermediate Repr. Geometric Features -CLODs -HapticTextures Contact Forces - Force Model for Texture Rendering Transparency - Implicit Integration - Linearized Contact

72 Future Work Fast, more general deformations, cutting Devices and interaction paradigms for Full body interaction with general environments

73 Acknowledgements Profs. Ming C. Lin, Markus Gross Profs. Manocha, Taylor, Snoeyink, and Brooks Stephen Ehmann, Young Kim, Nitin Jain, Avneesh Sud, Naga Govindaraju, Mark Foskey, Bill Baxter, Nico Galoppo, Denis Steinemman, Tanja Kaeser, Peter Leskovsky Profs. Roberta Klatzky and Susan Lederman ETH and Gamma UNC

74 Acknowledgements CoMe NCCR of the Swiss NSF Govt. Basque Country, UNC Dept. Alumni (Fellowships) Intel, US ARO, NSF, ONR

75 References [Colgate et al. 1995] Issues in the haptic display of tool use. IROS. [Galoppo et al. 2006] Fast simulation of deformable models in contact using dynamic deformation textures. SCA. [Johnson et al. 2005] 6-DoF haptic rendering using spatialized normal cone search. IEEE TVCG. [Klatzky and Ledermann 2002] Perceiving texture through a probe. Touch in Virtual Environments, by McLaughlin, Hespanha and Sukhatme. [McNeely et al. 1999] Six degree-of-freedom haptic rendering using voxel sampling. SIGGRAPH. [Ortega et al. 2006] A six degree-of-freedom god-object method for haptic display of rigid bodies. IEEE VR. [Otaduy and Lin 2003a] Dual hierarchies for multiresolution collision detection. SGP.

76 References [Otaduy and Lin 2003b] Sensation preserving simplification for haptic rendering. SIGGRAPH. [Otaduy et al. 2004] Haptic display of interaction between textured models. IEEE Visualization. [Otaduy and Lin 2004] A perceptually-inspired force model for haptic texture rendering. APGV. [Otaduy and Lin 2006] A modular haptic rendering algorithm for stable and transparent 6-DOF manipulation. IEEE TRO. [Otaduy and Gross 2007] Transparent rendering of tool contact with compliant environments. WHC. [Yoon et al. 2004] Fast collision detection between massive models using dynamic simplification. SGP.