Dr. Pierre-Frédéric Villard Research Associate

Transcription

1 Advanced Computer Graphics and Visualization Lecture 15 Dr. Pierre-Frédéric Villard Research Associate imperial. ac. uk,, St Mary s s campus Division of Surgery, Oncology, Reproductive Biology and Anaesthetics

2 Advanced Graphics and Introduction device characteristics rendering characteristics Collision detection and collision response Surface Effects Example: liver biopsy simulation Summary and Conclusions Hands-on session 2/52

3 Interaction Meaning to touch from the Greek haptesthai Refers to touch interactions (physical contact) for the purpose of perception or manipulation of objects 3/52

4 Goal of haptic rendering Enable a user to touch, feel and manipulate virtual objects Enhance a user s experience in a synthetic environment Provide a natural intuitive interface Real-time haptic sculpting (caltech) 4/52

5 s - Interface Design Useful for object manipulation Can be intimidating Mimic real world interaction Could be: - Tactile spatial distribution of forces - Kinesthetic own body position and motion BSc VR Surgical Simulation Slide 5 5/52

6 Examples In the past decade we have seen an enormous increase in interest in the science of haptics. Disciplines ranging from: robotics, telerobotics, computational geometry, psychophysics, cognitive science, and the neurosciences. Commercial products are already available on the market 6/52

7 Examples Phantom Omni from Sensable 6DOF Delta from Force Dimension 6DOF Phantom Premium 1.5 Falcon from Novint Mantis from Mimic 7/52

8 Golf Training GRADENER AND SASCH ORLIC, PGA GOLF PROFESSIONAAL 8/52

9 Medical Training simbionix Xitact (mentice) 9/52

10 Tactile Interfaces example (1/2) New Tactile Arrays An array of vibratory transducers that mimic complex tactile sensations to be interpreted by skin receptor systems Exeter Fingertip Stimulator 100 pins over 1 cm 2 Each pin has piezoelectric actuator Frequency range of Hz (Enough to simulate wide range of sensations) STReSS tactile display 10/52

11 Tactile Interfaces example (2/2) Other Types Pain Station Produces sensation of thermal and mechanical pain AMA Cologne, Germany Interface based on electrorheological fluids Hull Univ. - UK DataGlove modified with pneumatic actuators for force feedback - TNEL, Japan DataGlove senses finger position - VPL 11/52

12 Ultimate workstations Immersion 12/52

13 Comparison of rendering types Visual rendering Real time Scenegraph based Need: Geometry Material parameters BRDF texture Observer position Light position Light characteristics Updatable with displacement/deformation 13/52

14 Advanced Graphics and Introduction device characteristics rendering characteristics Collision detection and collision response Surface Effects Example: liver biopsy simulation Summary and Conclusions Hands-on session 14/52

15 Ground location It defines the relationship between the haptic device and the user: Ground based Finger 15/52

16 Mechanical behaviour It defines the mechanical process used to produce a feedback: Impedance based Admittance based 16/52

17 Number of degree of freedom It defines the number of dimensions characterizing the possible movements or forces exchanged between device and operator Example: 1 DOF: 3 DOF: 6 DOF: 17/52

18 Other characteristics Low back-drive inertia and friction Minimal constraints on motion imposed by the device kinematics Symmetric inertia, friction, stiffness, and resonate frequency properties Balanced range, resolution, and bandwidth of position sensing and force reflection Ergonomics 18/52

19 Advanced Graphics and Introduction device characteristics rendering characteristics Collision detection and collision response Surface Effects Example: liver biopsy simulation Summary and Conclusions Hands-on session 19/52

20 Aim of the haptic rendering User time Display Deformation computation Collision detection 20/52

21 Avatar definition Avatar = virtual representation of the haptic interface through which the user interacts with the virtual environment Depends on what s being simulated The operator controls the avatar s position 21/52

22 Collision Detection/Response Position Orientation Contact Information Collision Detection Object Database Geometry Force Torque Collision Response Material 22/52

23 Collision Detection The algorithms combine the information to obtain the positions in Cartesian inside the virtual environment. The algorithm uses position information to find collisions The algorithm reports the resulting degree of penetration or indentation. 23/52

24 Collision Response The algorithm computes interaction forces between objects involved in a collision. The algorithm sends interaction forces to the control algorithms 24/52

25 Hardware limitation It prevent haptic devices from applying the exact force computed by the force-response algorithms to the user Minimizes the error between ideal and applicable forces Discrete-time nature of the haptic-rendering algorithms often makes this difficult 25/52

26 A basic loop 1. Sample the position sensors at the haptic interface device joints 2. Compute the avatar s position inside the virtual environment. 3. Execute the collision-detection algorithm 4. Execute the force-response algorithm. 5. Apply forces while maintaining a stable overall behaviour 26/52

27 About the servo rate A 1-KHz servo rate is common Mapping with other models (collision, ) 27/52

28 Advanced Graphics and Introduction device characteristics rendering characteristics Collision detection and collision response Surface Effects Example: liver biopsy simulation Summary and Conclusions Hands-on session 28/52

29 29/52 Advanced Graphics and First illustration ) ( ) ( ) ( s u s u s u z z y y x x d + + = How to check if there is a collision How to compute the force Sphere (x s, y s, z s ), R User (x u, y u, z u ) = > = s u s u s u z z y y x x d N N d R k R d F 1 otherwise ) ( 0 r r

30 How to model the contact zone several models can be used. point-based model (torque is not considered) line-based model 6 DOF haptic objects-based model interface is needed F T F T F a point a line segment a 3D object 30/52

31 Penalty methods Algorithm: Subdivide the object volume Associate a sub-volume for each surface Determine force from penetration depth Work well for simple objects 31/52

32 Limitations of penalty methods Force discontinuity Pop-Thru of thin Object 32/52

33 Constrained based method Constrain a virtual proxy of the haptic interface to remain on the surfaces: 33/52

34 34/52

35 minimize x p subject to nˆ nˆ nˆ T 1 T 2 T m x 0, x 0, M x 0, 35/52

36 Advanced Graphics and Introduction device characteristics rendering characteristics Collision detection and collision response Surface Effects Example: liver biopsy simulation Summary and Conclusions Hands-on session 36/52

37 Surface properties Normal Stiff contact Friction Texture 37/52

38 Force Shading Render object surfaces as smooth and continuous, even when the underlying representation is not Analogous to Phong Shading 38/52

39 Stiff Contacts Happening when user contact the surface Perpendicular to the surface Common basis Spring force: f = Kx To perceive a sufficiently stiff contact Spring force and viscous damping force: f = Kx + Bv 39/52

40 Friction Happening when user stroke the surface The lateral force, opposite to motion The function of the coefficient of friction and normal force Coulomb friction: static and dynamic friction 40/52

41 Texture Varied ways to represent and display texture Deterministic textures Stochastic Models recordings 41/52

42 Advanced Graphics and Introduction device characteristics rendering characteristics Collision detection and collision response Surface Effects Example: liver biopsy simulation Summary and Conclusions Hands-on session 42/52

43 VR Simulator for Liver Biopsy DICOM DATA Interactive Segmentation Framework SIMULATOR Raw image data Labelled Segmentation Surface mesh Tetrahedral mesh Deformable Modelling Mesh Generation Pipeline 43/52

44 Advanced Graphics and rendering with H3D Open source API for haptic rendering Based on X3D file format Scene-graph descriptions programmable in C++: Field: data storage and functional behaviour Specializing the update function Node: build scene-graph and field container 44/52

45 Try the software 45/52

46 Advanced Graphics and Introduction device characteristics rendering characteristics Collision detection and collision response Surface Effects Example: liver biopsy simulation Summary and Conclusions Hands-on session 46/52

47 Summary We have seen: The concepts of haptic feedback with examples The characteristics of a haptic device (ground location, DOF, ) The steps done during a haptic rendering loop The collision detection more in details The surface properties An example of using haptic rendering in a VE surgical simulation 47/52

48 Comparison of rendering types Visual rendering Real time Scenegraph based Need: Geometry Material parameters BRDF texture Observer position Light position Light characteristics Updatable with displacement/deformation 48/52

49 Comparison of rendering types Visual rendering Real time Scenegraph based Need: Geometry Material parameters BRDF texture Observer position Light position Light characteristics Updatable with displacement/deformation rendering Real time Scenegraph based Need: Geometry Material parameters Elasticity roughness User force Avatar position Avatar geometry Updatable with displacement/deformation 49/52

50 Conclusion The haptic rendering allows adding sense of touch to the visualisation immersion A range of haptic devices are available with different specificities Many solution to handle contact and collision response exist 50/52

51 Advanced Graphics and Introduction device characteristics rendering characteristics Collision detection and collision response Surface Effects Example: liver biopsy simulation Summary and Conclusions Hands-on session 51/52

52 Demo Surface roughness Magnetic field 52/52

53 References K. Salisbury, F. Conti, F. Barbagli: rendering: introductory concepts in Computer Graphics and Applications, IEEE, vol.24, no.2, pp , March-April 2004 S.D. Laycock, A.M. Day: A Survey of Techniques in Computer Graphics Forum Vol. 26 Issue 1 Page 50 March 2007 F. Conti: s Interaction with Computer Modeled Objects in Stanford University lecture slides 53/52