Modeling the Virtual World

Transcription

1 Modeling the Virtual World Joaquim Madeira November, 2013 RVA /2014 1

2 A VR system architecture Modeling the Virtual World Geometry Physics Haptics VR Toolkits RVA /2014 2

3 VR object modeling Geometric modeling Kinematics modeling Physical modeling Model behavior Model management RVA /2014 3

4 VR object modeling RVA /2014 4

5 Geometric modeling for VR Surface shape Triangle meshes Different LODs Spline curves and surfaces Visual appearance Lighting and shading Texture mapping Multi-texturing RVA /2014 5

6 3D models for VR and AR Virtual / Augmented reality RVA /2014 6

7 Shape Define from scratch using VRML / X3D, OpenGL, VTK, Tedious; requires skill Obtain from CAD files or model databases Convert to compatible VR formats Use of existing models in manufacturing applications Create using a 3D digitizer or a 3D scanner 3D digitizer : stylus 3D scanner : tracker, cameras and laser RVA /2014 7

8 Shape RVA /2014 8

9 Visual appearance Scene illumination Local methods Handle one polygon / triangle at a time Flat / Gouraud / Phong shading Scene illumination Global methods How is the appearance of a given object influenced by other objects? Inter-reflections + Shadows More realistic results Mode demanding! Texture mapping RVA /2014 9

10 Flat shading vs Phong shading [Wikipedia] RVA /

11 Realistic lighting Vertex lighting of low polygon count surface - lights are diffuse - tessellated Vertex lighting of high polygon count surface - lights have realistic appearance - high computation load (from NVIDIA technical brief) RVA /

12 Ray-Tracing example RVA /

13 Another Ray-Tracing example RVA /

14 Ray-Tracing Diamond ring [ RVA /

15 Ray-Tracing Chemist s flask [ RVA /

16 Radiosity Without radiosity With radiosity RVA /

17 Texture mapping Map texture space coordinates to triangle vertices Then map to pixel coordinates Advantages? Increased scene realism Better 3D spatial cues Reduced number of triangles Increased frame rate

18 Texture mapping Smooth shading Environment mapping Bump mapping [Angel] RVA /

19 Textures Simulating Ray-Tracing [ Increased realism!! 11 light sources + 25 texture maps RVA /

20 Textures Define from scratch using interactive paint software Create bitmaps Custom textures from scanned or digital photos Convert to appropriate formats Use texture libraries Construction materials, cars, people, RVA /

21 Texture mapping RVA /

22 Texture mapping It glues an image to a polygon / surface Texture size has restrictions Texels Magnification vs minification Bilinear interpolation to assign colors to pixels RVA /

23 Tree represented as a texture Higher resolution model 45,992 polygons Texture 1 polygon, 1246 x 1280 pixels ( RVA /

24 Textures Normal texture Reflectivity texture Background texture Transparency texture Bump maps RVA /

25 Multi-Texturing Several texels can be overlaid on one pixel Texture blending cascade Application : Bump-Mapping Simulate lighting effects caused by surface irregularities Surface irregularities are coded as a texture No changes in model geometry No added computations!! RVA /

26 Bump-mapping Normal texture Multi-texture Bump texture RVA /

27 Light maps Realistic looking lighting can be done with 2D light maps Realistic and low polygon count Standard lighting map Light map texture overlaid 2-D texture on top of wall texture (from NVIDIA technical brief) RVA /

28 VR kinematics modeling Modeling and viewing transformations Object position Object hierarchies RVA /

29 Modeling and viewing transformations Homogeneous coordinates Matrix concatenation Create various model instances How? Position + Orientation + Size Change the position and aim of the virtual camera How? Fly-by, RVA /

30 Model instantiation Static objects Modeling matrix Moving objects Modeling matrix varies along the time RVA /

31 Tracking a virtual hand RVA /

32 Tracking a virtual hand Transformation matrices are concatenated to obtain the resulting motion If the object is grasped, its position does not change versus the hand The movement of the grasped object RVA /

33 VR kinematics modeling RVA /

34 Object hierarchies Models are structured as a hierarchy of parts Dynamics Different behavior for different model parts Parent-children relationship Higher-level vs lower-level The motion of a parent part is replicated by its children But not the other way around!! RVA /

35 Object hierarchies RVA /

36 Virtual hand Concatenate transformation matrices to obtain the motion of a fingertip Given by the glove tracker Given by the glove sensors RVA /

37 Virtual hand RVA /

38 VR physical modeling Add increased realism to the virtual world Integrate the physical characteristics of each object Weight Inertia Surface roughness Compliance : hard / soft Deformation mode : elastic / plastic RVA /

39 VR physical modeling Assign tasks to the haptics rendering pipeline What sequence? Computational load? RVA /

40 VR physical modeling RVA /

41 The rendering pipelines RVA /

42 The haptics rendering pipeline RVA /

43 Collision detection How to check for collisions between objects? It has to be done fast!! Exact collision detection Handle polygons / polyhedra Computational load!! Approximate collision detection First step! Use bounding boxes!! Easy to check if objects do not collide or might collide RVA /

44 Bounding boxes Bounding boxes for faster response times Variable size / orientation Fixed size / orientation Faster, but less precise! Axis-aligned bounding boxes AABB RVA /

45 Testing for possible collisions Do bounding boxes ovelap? Yes What to do? No OK! Test object pairs For every frame!?! RVA /

46 Undetected collision If the ball travels fast enough, the collision is not detected Increase rendering speed?!? Take into account scene coeherence RVA /

47 Two-stage collision detection Approximate detection Bounding boxes Identify / remove BBs that do not intersect Followed by exact detection Slower Various methods RVA /

48 The haptics rendering pipeline RVA /

49 Surface deformation Meshes Deforming a polygonal mesh Direct vertex manipulation Use the topological information Preserve mesh connectivity? Deformation propagation laws RVA /

50 Surface cutting Meshes Extreme case!! Contact force exceeds a given threshold Topological changes Co-located twin vertices separate and the cut enlarges Spring / damper laws Process propagates along the cutting path RVA /

51 Surface cutting Meshes RVA /

52 Force computation User interacts with 3D object surfaces Deformable or not User should feel reaction forces Reaction forces have to be Computed by the haptics pipeline Sent to the haptics display RVA /

53 Force computation Take into account Object physical and kinematics features Type of surface contact Kind of surface deformation Single-point contact Simplest case! Multiple contacts with a grasped object Haptic gloves RVA /

54 Force computation Immobile rigid objects Walls Mobile rigid objects Mobile elastic objecs Rubber balls Plastically deformable objects Soda cans RVA /

55 Single-point contact RVA /

56 Elastic deformation RVA /

57 Non-uniform hardness RVA /

58 Virtual push-button RVA /

59 Plastic deformation RVA /

60 Virtual Walls RVA /

61 The haptics rendering pipeline RVA /

62 Force smoothing Assumptions up to now: Single interaction point Frictionless contact The direction of the resistive force is the direction of the surface normal at the point of contact Unwanted discontinuities for polygonal meshes approximating curved surfaces Due to abrupt changes in normal direction RVA /

63 Force smoothing Objects feel faceted / tesselated Even if the graphics image does look smooth!! IDEA Extend the graphics shading approach to haptics Force shading changes the direction of the feedback force Polygonal surface models Simulate contact with smooth, curved surfaces RVA /

64 Force smoothing RVA /

65 Haptic meshes Fingertip-object contact A single haptic interface point (HIP) does not suffice!! More realistic modeling of the curvature of the fingertip and object deformation Haptic meshes RVA /

66 Haptic meshes RVA /

67 Haptic meshes RVA /

68 The haptics rendering pipeline RVA /

69 Force mapping RVA /

70 The haptics rendering pipeline RVA /

71 Haptics texturing Enhance the realism of the physical model of an object s surface Haptics texturing Similar to graphics textures adding realism to an object s visual appearance Add additional surface information Cold / warm Smooth Slippery RVA /

72 Haptics texturing RVA /

73 Haptics texturing TIME Tech October 09, 2013 Disney Just Figured Out Virtual Textures, Letting You Feel Bumps and Ridges on a Touchscreen YouTube: Tactile Rendering of 3D Features on Touch Surfaces RVA /

74 Behavior modeling Up to now we have been modeling Shape Visual appearance Kinematics Physical properties And assuming that each object is controlled by the user How to model behavior that is independent of a user s actions? RVA /

75 Behavior modeling Critical in large simulation environments where users cannot control all interactions taking place Interactive objects Agents RVA /

76 Behavior modeling Thalmann et al. (2000) distinguish three autonomy levels Guided / Programmed / Autonomous RVA /

77 Object behavior Model object behavior by accessing external sensors Such virtual objects are independent from the user s actions Example Virtual office Clock Access VR engine system time Thermometer Updated through a temp. sensor RVA /

78 Virtual human behavior Behavior model composed of Perception Emotions Behavior Actions Perception makes the agent aware of his / her surroundings RVA /

79 Agent behavior RVA /

80 Reflex behavior RVA /

81 Emotional behavior RVA /

82 Crowds behavior Emphasize group (rather than individual) actions Guided crowd User needs to specify path points, for instance Autonomous crowd Group perceives environment Group decides how to reach goal RVA /

83 Model management Rendering complex models / scenes / environments Maintain constant frame rates and simulation quality Allow interactivity Heavy computational load!!! How to address? RVA /

84 Addressing computational load LODs Close vs far objects Adaptive LOD Cell segmentation Architectural walk-throughs Lighting and bump mapping RVA /

85 Reference G. C. Burdea and P. Coiffet, Virtual Reality Technology, 2 nd Ed., Wiley-IEEE Press, 2003 Thanks are due to G. Burdea and P. Coiffet for making their slides available on the Web. RVA /