Overview. Agents and Avatars. Why IVAs? IVAS. Agents and Avatars Believability v naturalism Building the body H-anim Moving and sensing.

Transcription

1 Overview Agents and Avatars Ruth Aylett Agents and Avatars Believability v naturalism Building the body H-anim Moving and sensing IVAS Why IVAs? Intelligent Virtual Agents (IVAs) Also: Synthetic characters Embodied conversational characters (ECAs) Virtual humans BUT do not have to be humanoid Embodied and autonomous Require a control architecture - or agent mind Insides v outsides: combining AI with graphics Inhabit a virtual environment Adding life to a VE Animals, birds, insects Crowds E.g. students in the virtual campus Increase sense of presence As a guide or teacher Front-end to embedded knowledge in a VE As a character in a story Computer games: Creatures, the Sims Interface agents On web pages to make them more human Personal representative Sales rep 1

2 Why IVAs? - 2 As virtual actors Avatars Instead of extras Immersive Education Media Stage Snow Crash (Neil Stephenson 1992) - A graphical representation of yourself As part of a simulation Hostage release training Battlefield medical training Scientific investigation Buildings evacuation Testing for disability friendliness Ecology and animal behaviour Hindu Sanskrit Term - Representation of a Deity in visible form VE representation of the user Embodiment need not be humanoid Driven by the user So NOT autonomous A mapping rather than a control problem Video avatars 2

3 Creating a video avatar (ALIVE, MIT) magic video mirror with back-projection video image mixed with computer generated overlay unencumbered interaction between human visitor and virtual character based on position, postures, and gestures main focus on virtual presence Believability Term introduced by Joe Bates of the OZ group at CMU in the 1990s Combined art and technology Very hard to define A willing suspension of disbelief? Seem like real characters? The illusion of life? Willingness to attribute an internal state Ascribe intentionality Believability and Naturalism Are they the same thing? Graphics people seem to think so Is Mickey a real mouse? Is he a believable character? The uncanny valley The uncanny valley Work by a Japanese researcher, Mori Acceptability v naturalism Goes -ve as becomes nearly human SEE: 3

4 The problem of expectations Humans have hard-wired expectations Used to interpret inter-personal behaviour Fundamental social skill Need to invoke this very carefully Acceptability of movement Lip sync a key problem Interactive responsiveness Memory of interaction a key issue Games create a real problem with instant rewind Believability issues Building a body Consistency of character Use of 3D modelling package Movement, language, appearance Consistency with VE also an issue Creation of skeleton Polygonal body Plausible skills 3ds Character Studio; Poser Required for animation later But overall count must be low for real-time interaction polygon body Physical, social, knowledge Responsiveness Notice and repair errors Texture to cover body One body, many textures? 4

5 Capturing the user The Dome TalkZone Booth Taking the model home User interface Avatar Example 2. Enter Visitor Name 3. Generates Avatar Card 1. Checking the facial feature extraction Skate Boarding Scene Disco Dancing Scene 5

6 Stage 1: Image Capture Stage 2: Deform the Mesh Generic Avatar Polygonal Structure 1500 Polygons Change Data: Deforms the Mesh Texture Images X 4 Silhouette Images X 4 Stage 3: Apply the Textures The touch up process Apply Textures to Deformed Mesh 6

7 What sort of motion? Moving the body Walking around Jumping, running, swimming Human and non-human Picking things up Agent-agent interaction Gesture Talking heads Facial muscles, lip synch Group Movement Crowds, flocks, herds Animation Drawing on cartoon animation Motion capture Drawing on gait analysis and then film PBM Drawing on robotics Analytically calculated Learned Animation Self-animation Time consuming for good results Requires artistic skills The main character, Woody, in Toy Story had: 700 degrees of freedom (200 for face and fifty for the mouth) 150 people (at Pixar) would generate 3 minutes of animation a week Can it be standardised? Poses new problems: Parametrisable animation very desirable E.g a walk that could be reused with different stride length or foot height Melding and combining of animations on the fly Not just a morphing problem starting position? Extra actions inserted by character 7

8 Standardising animation? Animation depends on the structure of skeleton that is being animated Can we standardise a humanoid skeleton? H-anim just such an attempt Originally in the context of VRML See H-anim A set of standard components Humanoid: root of a figure Joint: attached using transform specifying current state of articulation plus geometry associated with attached body part Segment: specifies attributes of physical links between joints Site: where can add semantics Displacer:range of movement allowed for object in which embedded H-anim And in VRML.. DEF hanim_l_hip Joint { name "l_hip" rotation center children [ DEF hanim_l_thigh Segment { name"l_thigh" children Shape { appearance Appearance { material USE Pants_Color } geometry IndexedFaceSet { 8

9 Motion capture SIMON The Signer Electro-magnetic or by camera Produces the most natural results Extensively used in film for graphical extras Can be used on avatars very successfully to transmit user movement to their graphical representation Performs deaf-signing in real-time, over-laid on normal TV image Uses Teletext subtitle signal accompanies many broadcast programme Even more problems for self-animation Harder to parametrise Worse combining/melding problems Linguistic translation of subtitle content into a popular sign language Presented through on-screen, animated virtual human character Runs at client end SIMON The Signer - 2 A professional signer prepared for a motion capture session Dictionary of signed words for look up of accompanying physical movement, facial expressions and body positions, stored as motioncapture data Physical moves can be called in any order with interpolation to create(reasonably) smooth, natural looking signing sequences However a big acceptance problem Physically-based modelling Extending robot motor control Forward kinematics If you move joints so much Where does the end effector go? Inverse kinematics If you want the end effector at x,y,z How much should which joints move? Computationally demanding and often not very naturalistic But the most flexible option Animation blending often added 9

10 Fish example - spring-mass model Learning to move Use of AI learning algorithms Move Evaluate and score Keep movements which work well Karl Sims Blockies Early 1990s Terzoplolous - Fish Learning Virtual sensors Local v global interaction Global - read from world data-structures Local - sense the environment Global approach very common In most computer games Efficient, easy to test 10

11 Advantages of local sensing Scales well Not affected by global size of environment Agent has independence of environment Up to a point Makes for believability Agent perceives what it should Can t see you round corners Emergent complexity TeleTubby Sensors Forward ray-tracing sensor is seven meters and sweeps 45 o, five times/sec One vector sensor directed vertically downwards. Intersection with the ground is continually being detected All these sensors are attached to the geometry of the agent Virtual Vision How it s done Take description of 3D scene Plus camera position Produce 2D pixel array Colour and intensity Agent reconstitutes scene Using eye separation for depth 11

12 Processing Attentional focus Cuts down processing Directable eyes Look for interesting objects: orange pixels Known as active vision Variable resolution High-res at centre of field Sensing by message passing Requires an architecture that distributes events as messages Concept of locale What is local for local sensing? Semantically determined: e.g a room Scenegraphs not set up for this style of message passing Think of routing in VRML for example Pedagogical Agents - STEVE VET Architecture Runs the simulation Does the 3D rendering & detect interactions TTS Engine 12

13 Steve's Architecture Specific modules to steve general cognitive architecture that can model expert performance as well as novice behaviour. 13