Agents and Avatars 2. Ruth Aylett

Transcription

1 Agents and Avatars 2 Ruth Aylett

2 Overview- Crowds Speech and Expressive behaviour Embodied conversational characters Creating autonomy Scripting Architectures

3 Crowds and flocking Interactions among members of a group Local neighborhood Reynolds Boids Originally birds or fish Complex behaviour determined by simple rules

4 Separation: Boid Avoidance

5 Alignment

6 Cohesion

7 Motor Control Steering Force Integrate to determine acceleration Thrust determines speed Lateral Steering Force determines direction

8 Boid Object Representation Point Mass Vehicle Mass Position Velocity Orientation Constrained to align with velocity Force and Speed Limits (No moment of inertia)

9 Same maths as point objects acceleration = steering_force / mass Given f = ma velocity = velocity + acceleration Over a unit time period Generically v = v x at position = position + velocity Over a unit time period Generically s = s + vt

10 Seeking and Fleeing Aim towards target Desired_velocity = Kp (position target) Steering = desired_velocity velocity Seeking and Fleeing Applet (Reynolds)

11 Pursuing and Avoiding Target is another moving object Predict target s future position Scale prediction time, T, based on distance to object, D c T=D c Pursuing and avoiding applet (Reynolds)

12 More Behaviors Evasion Like flee, but predict pursuer s movement Arrival Like seek, but stop at target Applet (Reynolds) Obstacle Avoidance Repulsive force Aim to boundary Adjust velocity to be perpendicular to surface normal

13 Do People Flock? Social psychologist s report that people tend to travel as singles or in groups of size 2 to 5. Controlling Steering Behavior for Small Groups of Pedestrians in Virtual Urban Environments Terry Hostetler, Phd dissertation, 2002

14 Characteristics of Small Groups Proximity Coupled Behavior Common Purpose Relationship Between Members

15 Moving Formations Pairs: Triples: Side by side Triangular shape

16 Stationary Formations Moving pair approaches stationary triple Stationary quintuple formed

17 Locomotion Model for Walking Two Parameters - because in 2D Acceleration Increase/reduce walking speed Combination of step length and step rate Turn Adjust orientation Heading direction for forward walking

18 Avoiding an Obstacle -- Trajectory walkway axis walkway axis ped 1 ped 1 ped 2 ped 2 Small look-ahead distance Large look-ahead distance

19 Interaction Between Pairs -- 1

20 Interaction Between Pairs -- 2

21 Interaction Between Pairs -- 3

22 Adding goals People are usually going somewhere Easy case: same goal Walking to a sports stadium Political demonstrations Harder case: multiple goals Campus traffic Most public spaces: streets, shops, stations

23 Talking Heads Human faces Thus high expectations: Blinking, random head movement Facial muscles Lip sync Rarely intelligent Entirely scripted in most cases No integration of speech production with intelligent architecture and NL generation

24 Moving the face - 1 Simple approach Define overlay frames with different facial expressions For example, six mouth shapes Use signal from TTS engine to pick a mouth shape This works OK with cartoon-like characters

25 Moving the face - 2 Visemes Mouth and lip position for each phoneme Correspond to a submesh in the geometry

26 Using visemes Find viseme for each phoneme Synchronise graphical change with sound

27 Catalan weather forecaster Complete talking character Weather forecasts have predictable content Still have to match content and behaviour <demo via browser>

28 Embodied conversational characters Beyond talking heads Key difference is interactivity Use of dialogue theory Focus on body language - expressive behaviour Relating graphical to language behaviour

29 Expressive facial behaviour Drawing on psychology: FACS (Facial Action Coding System) Developed by Ekman & Friesen (1978) Distinguishes among 44 Action Units Each action unit (AU) describes a separate, visible, and muscle-based facial change

30 Duchenne de Bologne French anatomist of 19thC In 1860s used electric currents on facially paralysed subject Showed which muscles go with which expressions

31 The Duchenne smile Associated with expression of happiness AU12 (Lip Corner Puller) Plus AU6 (Cheek Raiser)

32 Mapping onto mpeg-4 Defines FAPs Facial action parameters Set of feature points Model calibration Morphology parameters Expression parameters

33 Coordinating with language Use of Affective Performance Markup Language (APML) <performative type="inform"><theme> As far as <emphasis x- pitchaccent="lplushstar"> vitamins </emphasis> are concerned <boundary type="lh"/></theme></performative> <rheme><emphasis x-pitchaccent="hstar"> research </emphasis> has shown <boundary type="lh"/></rheme> <performative type="inform"> <rheme>that eating<emphasis x- pitchaccent="hstar">the recommended </emphasis>levels of vitamin <rheme affect="sorry-for">can have <emphasis x pitchaccent="hstar"> beneficial </emphasis><emphasis x-pitchaccent="hstar">effects </emphasis>for your <emphasis x-pitchaccent="hstar">appearance </emphasis> <boundary type="lh"/> and <emphasis x- pitchaccent="hstar"> health</emphasis><boundary type="ll"/> </rheme></performative>

34 Incorporating gesture Additional markup to control gestures in synch with other expressive behaviour

35 Generating expressive behaviour Setting the internal state to be expressed Low-level accounts Focus on behaviour (non-symbolic) Primitive emotions Brain limbic system: physiological integration Canamero, Velasquez, Damasio High-level accounts Focus on cognitive appraisal Complex taxonomy of emotions Cortex: cognitive integration Ortony, Clore and Collins; many applications Frijda; Lazarus; Sherer

36 Modelling emotion Appraisals Assessments of events, actions, objects Valence Whether emotion is positive or negative Arousal Degree of physiological response Implementing appraisals Domain-specific rules Probability of impact on agent s goals

37 Emotion theory Ortony, Clore and Collins (OCC) Extremely widely used for agents Ortony, A; Clore, G. & Collins, A The cognitive structure of emotions. Cambridge University Press Group Well-being Fortunes-of-others Prospect-based Attribution Attraction Well-being! Attribution (compound emotions) Types joy, distress happy-for, gloating, resentment, sorry-for hope, satisfaction, relief, fear, fears-confirmed, disappointment pride, admiration, shame, reproach love, hate admiration + joy " gratitude Reproach + distress " anger Pride +joy " gratification Shame +distress " remorse

38 OCC Model - more detail Group Specification Name & type Well Being Fortunes of Others Prospectbased Emotions defined in terms of situations, goals, and others Appraisal of situation as event Appraisal of situation as event affecting another Appraisal of situation as a prospective event Joy - pleased about event Distress - displeased about event Happy-for: pleased about an event desirable for another Resentment: displeased about an event desirable for another Hope: pleased about a prospective desirable event Fear: displeased about a prospective undesirable event

39 Autonomy An independent sense-reflect-act cycle: implies An agent-based action-selection mechanism Self-animation in real-time: cannot be pre-rendered Local sensing Virtual robotics Not a natural graphics view Global manipulation of polygons Hierarchical scene-graph

40 Scripting Typical of NPCs in computer games What representational content is needed? Can just invoke named animations What parameters? Depends on animation approach

41 Parametrised Action Representation Badler: symbolic term -> movement Supports natural language interaction Named actions: walk, jump, run Adverb modification: quickly, tiredly, enthusiastically Laban Movement Analysis From choreography Effort and shape components to characterise movements

42 Badler s PARs

43 Scripting with Finite State Automata Tennis game FSM sequence

44 Limitations of scripting Becomes predictable Either independent of local sensing Or single FSM transition stimulus Limited ability to use internal state Interaction memory Affective state Lacks sequencing flexibility Sequence is hardwired

45 Low-level architecture for a fish Functional Architecture

46 High-level architecture

47 Using game engines Offer some support for NPCs More facilities than a scenegraph Attaching animations to graphical bodies Imports from popular 3D packages like character studio Disadvantages Proprietary; oriented to scripting Hard (in some cases impossible) to implement sensing Fixed paths quite often Often focused on shooting

48 Gamebots Loosely coupled architecture based on Unreal Tournament CMU Agent mind outside of UT Defined interface: message types and content Javabots gives java interface

49 The mechanism Gamebots and TCP sockets Virtual robot Unreal Server Command Sensor data Gamebots Command Sensor data Robot Controller TCP Socket (response to the command; send out sensor data) Command Sensor data Command Sensor data Unreal Client(s) Every robot possesses one TCP socket Exchange data through the TCP socket Unreal Client(s) can connect to the server at anytime Robot Controller

50 Communication protocol Gamebots protocol Format: data_type {segment1} {segment2} data_type: the type of the data in upper case characters Examples: INIT, STA, SEN, DRIVE etc. segment: a list of name-value pairs separated by space Examples: {Location 100,200,300}, {Name Left Range 800.0} etc. Example INIT {ClassName USARBot.ATRVJr} {Location 200,600,-450} Two types of data Message (from the server) Command (from the client)

51 Communication protocol - 2 Messages State message: the robot s current state STA {Time t} {Camera pitch,yaw,roll} {Zoom fov} {Attitude pitch,yaw,roll} {Location x,y,z} {Velocity x,y,z} {LightToggle bool} {LightIntensity int} {Battery float} Sensor message: the sensor data Sonar Sensor SEN {Type Range} {Name string Range number} {Name string Range number} Laser Sensor SEN {Type RangeScanner} {Name string} {Location x,y,z Rotation pitch,yaw,roll} {Range r1,r2,r3 } Human Motion Detection SEN {Type HumanMotion} {Name string} {Prob float} Sound Sensor SEN {Type Sound} {Name string} {Loudness float} {Duration float} Geometry message: sensor s geometry information GEO {Type string} {Name string Position x,y,z Direction pitch,yaw,roll} {Name string Position x,y,z Direction pitch,yaw,roll} Configuration message: sensor s configuration information CONF {Type string} {Name Value} {Name Value}

52 Communication protocol - 3 Command Spawn robot INIT {ClassName robot_class} {Name robot_name} {Location x,y,z} robot_class: the class name of the robot. Example: USARBot.P2AT, USARBot.P2DX etc. robot_name: the robot s name. Can be any string. x,y,z: the start position of the robot. For different arenas, we need different positions. Example INIT {ClassName USARBot.P2AT} {Location 312,-600,-305}