Daniel Thalmann EPFL - VRlab. Real-Time Autonomous Crowds. Dongguk University, Seoul, May VRlab

Transcription

1 Daniel Thalmann EPFL - VRlab Real-Time Autonomous Crowds Dongguk University, Seoul, May 2006 VRlab 1

2 Director: Prof. Daniel Thalmann About 25 people 5 EU projects Swiss National Research Foundation NCCR CO-ME Areas of research High-level Motion Control Models Walking, gait Automatic Grasping Models from Motion Capture Modeling of complex tasks Standards: MPEG-4, HANIM 2

3 Areas of research Behavioral animation, autonomous agents Groups and crowds Smart Objects Areas of research Body deformations Biomechanics models (medical) Health Emergency 3

4 Areas of research Networked VR VR interaction Haptics Augmented reality sense of presence Believable interfaces Graphics rendering Haptic rendering Acoustic rendering Study of human factors Personal interfaces (Emotional content) Address full range of population: from tech gurus and youngsters to elders and handicapped 4

5 How to model groups and crowds? What is a crowd? A large group of individuals in the same physical environment sharing a common goal and may act in a different way than when they are alone (Benesh, 86 and Roloff, 81) Motivation Why do we need virtual crowds? Real worlds: crowds are ubiquitous Non-real time applications: (feature films, commercials, cut-scenes of games) crowds used more and more, usually to increase epic dimensions Real-time applications: (games, training simulations) crowds are still rare, most interactive virtual worlds are ghost towns 5

6 Introduction Existing application areas for crowd simulations: Entertainment industry (animation production, computer games) Training of police & military (demonstrations, riots handling) Architecture (planning of buildings, towns, visualization) Safety science (evacuation of buildings, ships, airplanes) Sociology (crowd behavior) Physics (crowd dynamics) State-of-the-Art Flocking systems (Boids) (Reynolds, 1987) Procedural animation of flocks (Girard, 90) Film Eurythmy Maintain proper position and orientation: ~Avoid collisions ~Match velocities of neighbours ~Move towards the centre of flock 6

7 Crowd Evacuation Simulators Model movement of large number of people in usually closed, well-defined spaces like: inner areas of buildings (Simulex, Thompson & Marchant 95), subways (Hareesh et al. 2000), ships (Klüpfel et al. 2000) airplanes (Owen et al. 98). LEGION Early prototype of Legion TM developed by Still (2000) for simulation/analysis of crowd dynamics (stadiums). Legion v.1.7,: every person treated as individual. Each individual scans local environment and chooses action to minimise effort. 7

8 Crowd Management Training Systems CACTUS (Williams 1995) to assist in planning and training for public order incidents. crowd groups and police units placed on digitized map and have probabilistic rules for their interactive behavior (DAG). Small Unit Leader Non-Lethal Training System (Varner 98) simulator for training US Marines crowds move within simulated urban environment along instructor-predefined pathways crowd profiles fanaticism, arousal state, prior experience with non-lethal munitions State-of-the-Art: 3D systems Bouvier ( ) avoid obstacles using attraction and repulsion forces analogous to physical electric forces. Higher level behavior modeled by transition networks Exodus 8

9 State-of-the-Art Significant Physics (Brogan and Hodgins, 1997) Two steps: 1) perception model determines creatures and obstacles visible to each individual 2) placement algorithm determines desired position for each individual. Simulated systems: groups of legged robots, bicycle riders... State-of-the-Art: Crowd rendering Tecchia and Chrysantou (2001) Medieval Total War Fast rendering (Image Based billboarding) - Lot of pre-processing - Static animations 9

10 State-of-the-Art: Movies Digital cloning of people using 3D characters (The Borrowers and Titanic, 1998) State-of-the-Art: Movies Fluid mechanics (Stam, 99; Witting, 99) Hybrid systems: physical forces (flow fluids, goals) and procedural rules (flocking behaviours). Antz and Bugs Life (1999) 10

11 State-of-the-Art: Movies Industrial Light and Magic Star Wars: A Jovial Crowd of Geonosians State-of-the-Art: Movies PDI/DreamWorks Shrek 2 11

12 State-of-the-Art: Movies Lord oftherings The Mummy The Lion King Games: Act Of War 12

13 ViCrowd Script control (Scripted behaviors ) Events and Reactions Crowd Information Crowd Behaviour External control (Guided behaviors) GI G B B GI B GI B Individual Information and behaviours Script Language (Before simulation) User intervention during the simulation Generated by ViCrowd S. Raupp Musse, D.Thalmann, Hierarchical Model for Real Time Simulation of Virtual Human Crowds, IEEE Transactions on Visualization and Computer Graphics, 2001, Vol.7, No2, pp Crowd formed by groups Group formed by agents Agents completely programmed or evolves according to behavioural rules contains information to be distributed among groups size variable, can change during simulation possible leader can be programmed, guided or autonomous communication, memory, react to events, etc. cannot be explicitly controlled applythebehaviourofthegroup possible leader can change of group can apply innate behaviours : goal seeking, follow groups specification, walk and apply actions, etc. 13

14 Emergency Situation Crowds Real-Time Crowds Concepts Daniel Thalmann Pablo de Heras Ciechomski Jonathan Maïm Julien Pettré Sébastien Schertenleib Barbara Yersin 14

15 Requirements & constraints Different from single agent simulations Conceptual differences need for variety: Variety of agents visualizations Variety of individual trajectories Variety of behavior Variety of animations Variety of reactions to the same situations Assets Managements Data-Driven architecture Crowd simulation sub-areas Crowd behavior generation How should a virtual crowd behave? Crowd motion control How should virtual entities move around and avoid collisions? Integration of crowds in virtual environments Which aspects of environment need to be modeled? Interaction How can an end-user interact with the crowd in real-time 15

16 Our system Crowd simulation for interactive virtual environments: Goal to reproduce real world scenarios with large number of virtual humans evolving in realtime History: 2 generations of simulation systems Year SGI Onyx2 mainframe - Up to several dozens of agents - Few frames per second Year 2005: - Consumer grade PC - Up to several thousands of agents - Shaders Rendering for Crowds - Greater variety in appearance and animation Our system Features: Multi-agent system Data-Driven Individual based model Behavior modeled by Hierarchical Finite State Machine More complex behaviors described with Python Scripts. Interation and Feedback about the agents s status 3D Shaders Rendering engine Multi-threaded Environment 16

17 System Architecture Crowd simulation system: Computation Layers PrioritizedTasks Fast Locality Query Behaviors Transition Visualization layer Representing Behaviors Crowd Rendering Engine Authoring Tools Creating and interacting with the crowds Feedback information about the simulations Crowd Component: Part of the VHD++ real-time framework vhdcrowdservice Simulated crowd of virtual humans able to move and interact within virtual environment A Virtual Human-based VR/AR Platform: VHD++ Developed by VRlab (EPFL) and MIRALab (Univ. Geneva) Is a multithreaded component-based system development Extendible through Plug-Ins (Crowds, AI, Sound ) Comes with a set of standard plug-ins for handling: Scenegraph (using OpenSceneGraph, Scripting (Python, Lua) Configuration Files (XML, Collada) for system and data management Multi-platform (Linux, Windows) Used in different domain like VR, AR, Edutainment, 17

18 VHD++: spectrum of applications vhd LIFEPLUS AR edutainment reconstruction of ancient life in Pompei (Italy) VR VHD++ AR vhd ERATO Reconstructing of crowds in ancient theaters games edutainment Social Phobia infotainment vhd STAR AR training of hardware maintenance professionals vhd JUST immersive VR health emergency personnel training Crowd rendering engine skeletal and billboard rendering LODs generated with OGRE3D fully dynamic animation -> IK possible greater variety in appearance Shader Integration 18

19 Rendering Management: Manager stores template humans and keeps record of all existing instances. Template humans: set of animations, textures and shaders. Skeletal deformed characters rendered first, then billboard models (front-to-back) Benefits: Benefits of our dual approach are: Fully dynamic animation We have facial animation detail levels Characters look fine from all camera directions and zooms, as opposed to a pure billboard approach Memory usage lower than with billboards 19

20 Crowd rendering engine Full integration into OpenGL triangle processing pipeline Colored spotlight Cartoon shading Specular highlights shading Crowd generation 3ds max exporting Pipeline for converting character and animation data to format usable by crowd rendering and animation engine Exported data: Mesh Texture UV coordinates Skeleton hierarchy Deformations bindings Animations 20

21 Crowd generation Mesh design: around polygons each Textures design: same texture mapping for different meshes; different characters by varying colors Designing Texture Variety (1/3) - Random colors for each body part is not good enough - Realistic results require constrained randomness 21

22 Designing Texture Variety (2/3) HSB Maps HSB space constrained to a 3D color space. Allow to deal with localized constraints. Eyes example : - Hue from 20 to 250; - Saturation from 30 to 80; - Brightness from 40 to 100. Designing Texture Variety (3/3) Alpha Map Layer Allow the artist to define the different body parts. Allow the application to modulate the color variety. Example of 1 mesh template with 4 different textures. HSB maps give the illusion of many different meshes and textures. 22

23 AI processing Goal: Every agent has its own behavior. Agent Management Chained Hierarchical State Machine State Machine Transitions described within Lua Metatables Agent s tasks are prioritized in order to avoid unnecessary checks. (E.g. updating every state at each frame) Scenario Authoring Designers can create their own behaviors using scripting interfaces Behavior engine Virtual human agent Visual 3D Mesh, Skeletal Animation (compatible with H-ANIM standard) Motor Actions Steering, Walking, Gestual Animations (Lookat, Face Animations ) Instinct Behaviors Scenario specific states defining psychological and physical attributes. High-Level Behaviors Motivation Based Behaviors. Authoring Real-time interaction with the virtual human allowing to prototype and test the system. 23

24 Crowds Navigation (1/4) Objectives Crowds One to thousands of people Common or individual goals Variety Spread people Individual solutions Interactivity Environment Uneven surfaces Multi-floored structure Automatic Unprocessed environment meshes No user s intervention Crowds Navigation (2/4) Principles Based on a pre-computed Navigation Graph Capture the environment Topology Delimit Navigable Areas (vertices) Out of obstacle Limited slopes Cylindrical bounding shape Local elevation map Connect areas with Passages (edges) Bounding shapes intersections Deterministic graph building method (based on Voronoï diagrams) 24

25 Crowds Navigation (3/4) goal Path Planning Iterative path search Provides multiple solutions (edge sequences) leading to the goal Captures several homotopy classes of solutions (i.e. separated by obstacles) From the shortest to the longest Path declination Transforms previous paths into way-point sequences Exploits passage widths to produce variety Individual parameter determines exactly where each passage is crossed init Crowds Navigation (4/4) Results Way-points following is scalable (smooth, linear or estimated trajectories) Data structure is compact, rich and memory-light Navigation graph computed deterministically in minutes Robust and versatile Inter-collisions are to be considered locally only 25

26 Crowd Motion control Displacement system Motion of agents between waypoints Activate Motor Actions (walking, running skeletalbased animations) Variety Walking attribute (velocity, speed ) Non-static waypoints through collision free corridors. Adaptive walking style using Motion Capture Data Characters motion using the navigation graph The promising approach: generate models from motion capture e.g. Walking model Our walking engine: driven by two highlevel parameters: speed and human size. 1. interpolation and extrapolation achieved in PCA spaces of each subject to generate new motion according to speed value. 2. time warping method allows to handle the human size parameter. 26

27 Creation of PCA space PCA technique transforms set of variables into smaller set of its linear combination, so as to account for most of variance of original dataset. PCA space computed with motion matrix M composed of various walking motion vectors θ from specific subject v. θ 0 : average vector of all n motion vectors of M. α=(α 1, α 2,, α m ) coefficient vectors e=(e 1, e 2,, e m ) Basic vectors: m first PC's of M, walking motion θ: Characteristics of the model We can extrapolate and interpolate realistic and correct walking motion from 0 km/h up to 10 km/h Beyond 10 km/h, undesired behavior occurs at skeleton level esp. legs) - Double support phase no longer ensured. Scaling size of human directly influences cycle frequency in order to preserve motion properties. For 2 humans with different size, smaller one walks with larger frequency for identical motion speed. 27

28 Crowd authoring No matter how good rendering or behavior, there should also be a good way of producing content for simulation More individuals more difficult to create unique and varied content Create and modify characters one by one is too laborious Procedural generation can create artifacts ( armylike appearance) Random distributions are hard to control Need human-in-the-loop to make creative decisions Scenario authoring CrowdBrush Goal is to give full creative power to designers using metaphors of artistic tools Works in WYSIWYG (What You See Is What You Get) mode, with a real-time view of the authored scene 28

29 Scenario authoring Brushes Tools with visual representation on the screen Affect crowd members in different manners: Create new individuals in the scene Change their appearance or behavior Creation brush Deletion brush Happy brush Sad brush Crowds in VR training Applications Training system for urban emergency situations Health emergency decision training Goal To reproduce training scenarios based on real world situations 29

30 Agent-based Case: Riot in the city Crowds in virtual heritage CAHRISMA Reconstructions of mosques Around 50 characters ERATO Reconstruction of theaters and odeons Around 1000 characters 30

31 Future We will see more and more people Faster CPUs & GPUs Multi-threaded Architecture (Multi-core CPUs) Better algorithms Content creation streamlining Convergence between non real-time and real-time domains 31