Modeling the Virtual World Joaquim Madeira November, 2013 RVA - 2013/2014 1
A VR system architecture Modeling the Virtual World Geometry Physics Haptics VR Toolkits RVA - 2013/2014 2
VR object modeling Geometric modeling Kinematics modeling Physical modeling Model behavior Model management RVA - 2013/2014 3
VR object modeling RVA - 2013/2014 4
Geometric modeling for VR Surface shape Triangle meshes Different LODs Spline curves and surfaces Visual appearance Lighting and shading Texture mapping Multi-texturing RVA - 2013/2014 5
3D models for VR and AR Virtual / Augmented reality RVA - 2013/2014 6
Shape Define from scratch using VRML / X3D, OpenGL, VTK, Tedious; requires skill Obtain from CAD files or model databases Convert to compatible VR formats Use of existing models in manufacturing applications Create using a 3D digitizer or a 3D scanner 3D digitizer : stylus 3D scanner : tracker, cameras and laser RVA - 2013/2014 7
Shape RVA - 2013/2014 8
Visual appearance Scene illumination Local methods Handle one polygon / triangle at a time Flat / Gouraud / Phong shading Scene illumination Global methods How is the appearance of a given object influenced by other objects? Inter-reflections + Shadows More realistic results Mode demanding! Texture mapping RVA - 2013/2014 9
Flat shading vs Phong shading [Wikipedia] RVA - 2013/2014 10
Realistic lighting Vertex lighting of low polygon count surface - lights are diffuse - tessellated Vertex lighting of high polygon count surface - lights have realistic appearance - high computation load (from NVIDIA technical brief) RVA - 2013/2014 11
Ray-Tracing example http://radsite.lbl.gov/radiance/book/img/plate10.jpg RVA - 2013/2014 12
Another Ray-Tracing example http://www.tjhsst.edu/~dhyatt/superap/samplex.jpg RVA - 2013/2014 13
Ray-Tracing Diamond ring [http://www.okino.com] RVA - 2013/2014 14
Ray-Tracing Chemist s flask [http://www.okino.com] RVA - 2013/2014 15
Radiosity Without radiosity With radiosity RVA - 2013/2014 16
Texture mapping Map texture space coordinates to triangle vertices Then map to pixel coordinates Advantages? Increased scene realism Better 3D spatial cues Reduced number of triangles Increased frame rate
Texture mapping Smooth shading Environment mapping Bump mapping [Angel] RVA - 2013/2014 18
Textures Simulating Ray-Tracing [http://www.okino.com] Increased realism!! 11 light sources + 25 texture maps RVA - 2013/2014 19
Textures Define from scratch using interactive paint software Create bitmaps Custom textures from scanned or digital photos Convert to appropriate formats Use texture libraries Construction materials, cars, people, RVA - 2013/2014 20
Texture mapping RVA - 2013/2014 21
Texture mapping It glues an image to a polygon / surface Texture size has restrictions Texels Magnification vs minification Bilinear interpolation to assign colors to pixels RVA - 2013/2014 22
Tree represented as a texture Higher resolution model 45,992 polygons Texture 1 polygon, 1246 x 1280 pixels (www.imagecels.com) RVA - 2013/2014 23
Textures Normal texture Reflectivity texture Background texture Transparency texture Bump maps RVA - 2013/2014 24
Multi-Texturing Several texels can be overlaid on one pixel Texture blending cascade Application : Bump-Mapping Simulate lighting effects caused by surface irregularities Surface irregularities are coded as a texture No changes in model geometry No added computations!! RVA - 2013/2014 25
Bump-mapping Normal texture Multi-texture Bump texture RVA - 2013/2014 26
Light maps Realistic looking lighting can be done with 2D light maps Realistic and low polygon count Standard lighting map Light map texture overlaid 2-D texture on top of wall texture (from NVIDIA technical brief) RVA - 2013/2014 27
VR kinematics modeling Modeling and viewing transformations Object position Object hierarchies RVA - 2013/2014 28
Modeling and viewing transformations Homogeneous coordinates Matrix concatenation Create various model instances How? Position + Orientation + Size Change the position and aim of the virtual camera How? Fly-by, RVA - 2013/2014 29
Model instantiation Static objects Modeling matrix Moving objects Modeling matrix varies along the time RVA - 2013/2014 30
Tracking a virtual hand RVA - 2013/2014 31
Tracking a virtual hand Transformation matrices are concatenated to obtain the resulting motion If the object is grasped, its position does not change versus the hand The movement of the grasped object RVA - 2013/2014 32
VR kinematics modeling RVA - 2013/2014 33
Object hierarchies Models are structured as a hierarchy of parts Dynamics Different behavior for different model parts Parent-children relationship Higher-level vs lower-level The motion of a parent part is replicated by its children But not the other way around!! RVA - 2013/2014 34
Object hierarchies RVA - 2013/2014 35
Virtual hand Concatenate transformation matrices to obtain the motion of a fingertip Given by the glove tracker Given by the glove sensors RVA - 2013/2014 36
Virtual hand RVA - 2013/2014 37
VR physical modeling Add increased realism to the virtual world Integrate the physical characteristics of each object Weight Inertia Surface roughness Compliance : hard / soft Deformation mode : elastic / plastic RVA - 2013/2014 38
VR physical modeling Assign tasks to the haptics rendering pipeline What sequence? Computational load? RVA - 2013/2014 39
VR physical modeling RVA - 2013/2014 40
The rendering pipelines RVA - 2013/2014 41
The haptics rendering pipeline RVA - 2013/2014 42
Collision detection How to check for collisions between objects? It has to be done fast!! Exact collision detection Handle polygons / polyhedra Computational load!! Approximate collision detection First step! Use bounding boxes!! Easy to check if objects do not collide or might collide RVA - 2013/2014 43
Bounding boxes Bounding boxes for faster response times Variable size / orientation Fixed size / orientation Faster, but less precise! Axis-aligned bounding boxes AABB RVA - 2013/2014 44
Testing for possible collisions Do bounding boxes ovelap? Yes What to do? No OK! Test object pairs For every frame!?! RVA - 2013/2014 45
Undetected collision If the ball travels fast enough, the collision is not detected Increase rendering speed?!? Take into account scene coeherence RVA - 2013/2014 46
Two-stage collision detection Approximate detection Bounding boxes Identify / remove BBs that do not intersect Followed by exact detection Slower Various methods RVA - 2013/2014 47
The haptics rendering pipeline RVA - 2013/2014 48
Surface deformation Meshes Deforming a polygonal mesh Direct vertex manipulation Use the topological information Preserve mesh connectivity? Deformation propagation laws RVA - 2013/2014 49
Surface cutting Meshes Extreme case!! Contact force exceeds a given threshold Topological changes Co-located twin vertices separate and the cut enlarges Spring / damper laws Process propagates along the cutting path RVA - 2013/2014 50
Surface cutting Meshes RVA - 2013/2014 51
Force computation User interacts with 3D object surfaces Deformable or not User should feel reaction forces Reaction forces have to be Computed by the haptics pipeline Sent to the haptics display RVA - 2013/2014 52
Force computation Take into account Object physical and kinematics features Type of surface contact Kind of surface deformation Single-point contact Simplest case! Multiple contacts with a grasped object Haptic gloves RVA - 2013/2014 53
Force computation Immobile rigid objects Walls Mobile rigid objects Mobile elastic objecs Rubber balls Plastically deformable objects Soda cans RVA - 2013/2014 54
Single-point contact RVA - 2013/2014 55
Elastic deformation RVA - 2013/2014 56
Non-uniform hardness RVA - 2013/2014 57
Virtual push-button RVA - 2013/2014 58
Plastic deformation RVA - 2013/2014 59
Virtual Walls RVA - 2013/2014 60
The haptics rendering pipeline RVA - 2013/2014 61
Force smoothing Assumptions up to now: Single interaction point Frictionless contact The direction of the resistive force is the direction of the surface normal at the point of contact Unwanted discontinuities for polygonal meshes approximating curved surfaces Due to abrupt changes in normal direction RVA - 2013/2014 62
Force smoothing Objects feel faceted / tesselated Even if the graphics image does look smooth!! IDEA Extend the graphics shading approach to haptics Force shading changes the direction of the feedback force Polygonal surface models Simulate contact with smooth, curved surfaces RVA - 2013/2014 63
Force smoothing RVA - 2013/2014 64
Haptic meshes Fingertip-object contact A single haptic interface point (HIP) does not suffice!! More realistic modeling of the curvature of the fingertip and object deformation Haptic meshes RVA - 2013/2014 65
Haptic meshes RVA - 2013/2014 66
Haptic meshes RVA - 2013/2014 67
The haptics rendering pipeline RVA - 2013/2014 68
Force mapping RVA - 2013/2014 69
The haptics rendering pipeline RVA - 2013/2014 70
Haptics texturing Enhance the realism of the physical model of an object s surface Haptics texturing Similar to graphics textures adding realism to an object s visual appearance Add additional surface information Cold / warm Smooth Slippery RVA - 2013/2014 71
Haptics texturing RVA - 2013/2014 72
Haptics texturing TIME Tech October 09, 2013 Disney Just Figured Out Virtual Textures, Letting You Feel Bumps and Ridges on a Touchscreen YouTube: Tactile Rendering of 3D Features on Touch Surfaces RVA - 2013/2014 73
Behavior modeling Up to now we have been modeling Shape Visual appearance Kinematics Physical properties And assuming that each object is controlled by the user How to model behavior that is independent of a user s actions? RVA - 2013/2014 74
Behavior modeling Critical in large simulation environments where users cannot control all interactions taking place Interactive objects Agents RVA - 2013/2014 75
Behavior modeling Thalmann et al. (2000) distinguish three autonomy levels Guided / Programmed / Autonomous RVA - 2013/2014 76
Object behavior Model object behavior by accessing external sensors Such virtual objects are independent from the user s actions Example Virtual office Clock Access VR engine system time Thermometer Updated through a temp. sensor RVA - 2013/2014 77
Virtual human behavior Behavior model composed of Perception Emotions Behavior Actions Perception makes the agent aware of his / her surroundings RVA - 2013/2014 78
Agent behavior RVA - 2013/2014 79
Reflex behavior RVA - 2013/2014 80
Emotional behavior RVA - 2013/2014 81
Crowds behavior Emphasize group (rather than individual) actions Guided crowd User needs to specify path points, for instance Autonomous crowd Group perceives environment Group decides how to reach goal RVA - 2013/2014 82
Model management Rendering complex models / scenes / environments Maintain constant frame rates and simulation quality Allow interactivity Heavy computational load!!! How to address? RVA - 2013/2014 83
Addressing computational load LODs Close vs far objects Adaptive LOD Cell segmentation Architectural walk-throughs Lighting and bump mapping RVA - 2013/2014 84
Reference G. C. Burdea and P. Coiffet, Virtual Reality Technology, 2 nd Ed., Wiley-IEEE Press, 2003 Thanks are due to G. Burdea and P. Coiffet for making their slides available on the Web. RVA - 2013/2014 85