Virtual proxies in virtual worlds

Transcription

1 Virtual proxies in virtual worlds Doron Friedman Advanced Virtuality Lab Sammy Ofer School of Communications The Interdisciplinary Center, Herzliya LA, Feb. 2011

2 THE DAILY NEWS THE WORLD S FAVOURITE NEWSPAPER - Since 1879 THE ADVANCED VIRTUALITY LAB LA, Feb AVL is part of the Sammy Ofer School of Communications in the Intedisciplinary Center, Herzliya, Israel. It is headed by Dr. Doron Friedman, Research Fellows: Dr. Béatrice Hasler, Dr. Boris Oicherman, and Dr. Dan Drai, and a few research assistants and research students. AVL strives to understand virtuality in the beginning of the 21 st century from a broad multi-disciplinary perspective. This encompasses advanced virtual reality and humancomputer interface technologies, braincomputer interfaces, the neuroscience of mediated experiences, the psychology of online virtual worlds and online group collaboration, the philosophy of cyberspace, and the mathematics of cyberspace. Ea pro natum invidunt repudiandae, his et facilisis vituperatoribus. Mei eu ubique altera senserit, consul eripuit accusata has ne. Ea pro natum invidunt repudiandae, his et facilisis vituperatoribus.

3 People Alex Bass (BCI) Amit Bauer (AI) Anat Brovman (Culture studies) Ori Cohen (MSc CS, 3D animator) Ayal Donenfeld (Philosophy of cyberspace, HUJI PhD student) Dr. Dan Drai (Math, CS, and philosophy) Dr. Doron Friedman (Lecturer, Sammy Ofer School of Communications) Dr. Béatrice Hasler (media psychology) Daniel Korn (programmer & psy.) Nir Saar (resident hacker and filmmaker) Aidai Sidakmatova (Comm.) Peleg Tuchman (programmer) LA, Feb. 2011

4 Brain & body machine interfaces LA, Feb. 2011

5 EU FP7: VERE ( ) LA, Feb. 2011

6 Virtual worlds, real people D. Friedman, Y. Karniel, A. Lavie- Dinur, Comparing group discussion in virtual and physical environments, Presence: Teleoperators and Virtual Environments, 18(4), , D. Friedman, A. Steed, and M. Slater, Spatial social behavior in Second Life, Proc. Intelligent Virtual Agents LNAI 4722, Pelachaud et al. (eds), pages , Paris, France, September LA, Feb. 2011

7 Virtual Worlds, Real People The spectrum hypothesis (Friedman et al. 2009) The mapping principle (Williams, 2010) Examples of mapping principles (Hasler & Friedman, 2011) LA, Feb. 2011

8 Mapping, f: Φ-> V The visual saliency principle: Only factors that are visible in a dyadic interaction in virtual worlds influence interavatar distance, whereas invisible factors have no influence. (invisible factors include personality, gender,..!) LA, Feb. 2011

9 EU FP7 ( ): Beaming LA, Feb. 2011

10 The Beaming Proxy destination visitor avatar LA, Feb. 2011

11 LA, Feb. 2011

12 Being in two places at the same time LA, Feb. 2011

13 The Beaming Proxy destination visitor avatar LA, Feb. 2011

14 The Beaming Proxy destination proxy avatar LA, Feb. 2011

15 Generally user VE avatar AI LA, Feb. 2011

16 In our case user VE avatar LA, Feb. 2011

17 proxy AI VE avatar LA, Feb. 2011

18 Meet the Proxy Based on a true story: I was invited to answer some questions in world in the IMMERS(ED) 3D, The 2 nd National Workshop on Teaching in Virtual Worlds, Ulster, Northern Ireland (Oct 2010) Beaming proxy: The movie

19 chatbot mode Log in to specific area and mission Mission control

20 User friendly status Server messages

21 Running predefined scripts Script execution status

22 Behavioral cloning Behavioral model VE / Beaming Recorded activity logs controller P. Tuchman and D. Friedman, Virtual clones: Data-driven social navigation, submitted Jan

23 Simple social navigation: Approaching someone

24

25 Evaluation Automated setup participants: P1 (Male, age 24, num of samples: 50) P2 (M, 42, 15) P3 (F, 24, 20) P4 (F, 35, 9) P5 (F, 33, 98)

26 Recorded trajectories

27 Recorded trajectories cont

28 Recorded trajectories cont

29 Method 1: Probabilistic Markov models Trajectory => < t 1, ρ 1, θ 1, α 1, a 1 > < t k, ρ k, θ k, α k, a k > (ρ, θ) distance to target in polar coordinates α- relative gaze angles a R, L, F, B, F+R, F+L, B+R, B+L States: discretize ρ t x θ t x α t x a t-1

30 Generalization using tiling

31 Method 2: Codebook & nearest neighbor Preprocessing As before, each trajectory is treated as a set of stateaction pairs (s,a), s = <ρ t,θ t,α t,a t-1 >, a is a t We keep a codebook C of such pairs If necessary we use vector quantization (VQ) to reduce the size of the codebook Real-time The current state of the agent is converted into a vector s = <ρ t,θ t,α t,a t-1 > find pair (s,a) C s.t. s-s is minimal agent performs a

32 Distance metric ρ = ρ 1 - ρ 2 θ = 1 1 θ2 + cos( θ 2 ) x = <ρ, θ, α, a> Manhattan distance: d ( x, y) = x i i y i α = 1 1 α2 + cos( α 2 ) a = 1 if a 1 = a 2 0 if a 1 a 2

33 K-nearest neighbors arg max b j δ(a,b) = 1 if a = b 0 if a b

34 Adding learning from experience The trajectory planning may fail if there are not enough samples Generalization (such as tiling) is not a preferred option We apply the following algorithm: Sample N points s (vectors) in the state space, with uniform distribution For each state in s compute a trajectory t using the method mentioned above and codebook C t = <(s 1,a 1 ),,(s k,a k )>, where each s = <ρ t,θ t,α t,a t-1 > If t is successful then C = C { (s 1,a 1 ),,(s k,a k ) }

35 Proxy (automatically-generated) trajectories

36 Proxy (automatically-generated) trajectories

37 Are there individual differences? We cannot use statistical tests or classifiers due to large variability in the number of samples obtained from the different participants (this constraint is inherent to the proxy problem). Multidimensional scaling: given a distance matrix over I objects find I vectors x 1,,x I R N s.t. x i -x j δ i,j i,j I (i.e., en embedding of I objects into R N s.t. the distances among the objects are preserved as much as possible, although N <<I)

38 Are there individual differences? P1 vs P3: Multi-dimensional scaling with Euclidean distance

39 Are there individual differences? P1 vs P3: Multi-dimensional scaling with Manhattan distance d ( x, y) = x i i y i

40 Are there individual differences? P1 L P3 L P1 F+L P3 F+L

41 Thanks for your attention Doron Friedman LA, Feb. 2011