Abstract Media Spaces

Transcription

1 Abstract Media Spaces Rob Diaz-Marino CPSC 781 University of Calgary 2005 Outline What is Abstraction? Simple Media Spaces Abstract Media Spaces (AMS) Benefits and Drawbacks Methods for Abstracting Media Designing an AMS Summary 1

2 What is Abstraction? Guernica by Pablo Picasso Reaction to the bombing of Guernica in World War II. What is Abstraction? (2) Throwing away information [1] Realistic Abstract Photo Cartoon Art Piece Fact -Female -Blond Hair -Yelling -Emotion? -Female -Brown Hair -Yelling / Angry -Female -Yelling / Distressed -In a barn/cellar Abstraction -Image cropped -Unclear context -Part not drawn -No context -Lips, eyebrows, fingers, mouth, hair -Eyes on same side of head -Eyes in wrong place -Distorted body shape -Hair on back of head -Blue-gray color 2

3 Simple Media Spaces Transmission Video-In Video-Out Transmission Picture-In Picture-Out Transmission Audio-In and so on. Audio-Out Simple Media Spaces (2) Literal Transmission Input = Low degree of abstraction Some loss from original (reality) Capture limitations Compression bandwidth limitations Still perceptually equivalent 3

4 Why use Abstraction in a MS? To control information Preserve Privacy Shield sensitive details Reduce Distraction Eliminate unnecessary details Re-map awareness cues Reduce bandwidth needs Bob is at his computer Bob is not wearing clothes Bob is has a fruit-basket hat on his head Bob is yelling at his girlfriend on the phone Bob looks angry [1] Abstraction Someone is at the computer The person is mostly flesh-colored The person has something large on their head The person is speaking loudly No details can be seen on their face Drawbacks of Abstraction Loss of information Useful: Identity, Actions, Availability, etc. Incidental: Details, Emotional state, etc. Loss of understanding Unclear meaning Unclear context Bob is at his computer Bob is not wearing clothes Bob is has a fruit-basket hat on his head Bob is yelling at his girlfriend on the phone Bob looks angry Abstraction Someone is at the computer The person is mostly flesh-colored The person has something large on their head The person is speaking loudly No details can be seen on their face 4

5 Methods of Abstraction Simple Degradation Feature Extraction (Silhouetting) Video Simple Degradation Still resembles original Ex. Mike Boyle s s Video Filters [4] Blur Filter Video-Out Video-In Pixelation Video-Out 5

6 Audio Simple Degradation (2) Still resembles original Echo Audio Out Audio-In Muffling Audio Out Video Feature Extraction Vision Techniques Motion detection Presence detection Eye tracking Face tracking 6

7 Audio Feature Extraction (2) Ex. Smith et al s s Low Disturbance Audio [3] Speech Non-Speech Can still recognize voice Cannot understand words Similar to Blur Filter but for Audio! Extract Synthesize Audio-In Audio Out Text Feature Extraction (3) Ex. Syllable replacement Jim: Hi! How are you doing? Bob: Doing okay Jim: Are you busy? Bob: I m on the friggin phone!! Jim: Oh, sorry! Extract Jim: Bla! Bla bla bla blabla? Bob: Blabla blabla Jim: Bla bla blabla? Bob: Blablablablablabla!! Jim: Bla, blabla! 7

8 Media Translation Convert one media form to another No direct translation Feature Extraction Synthesizer Visualization, Sonification,, etc. Extract Synthesize Video-In Audio-Out Media Translation (2) Ex. AROMA [1] Peripheral awareness 8

9 Media Translation (3) Ex. Cambience (my thesis project) Inputs Web Cam (video) Feature extraction Motion detection, partitioning, thresholding, etc. s Audio volume, pan, etc. Translation Pitfalls Extreme abstraction No longer understandable Usable only as art piece Learning Curve Arbitrary mappings Users may need to see literal data [1] 9

10 Processing Designing an AMS Must be done in REAL TIME Can lower sampling rate to compensate Peripheral vs. Foreground Draw Inspiration Ambient Displays Visualizations Designing an AMS (2) 3 Architectures 1) Client-side processing 2) Server-side processing 3) Distributed processing 10

11 AMS Architectures (1) Client-Side Processing Transmit raw data privacy risk! High Bandwidth usage Low CPU for Server, high for Clients Server Client Extract & Synth Input Extract & Synth Extract & Synth AMS Architectures (2) Server-Side Side Processing Transmit synthesized media High Bandwidth usage High CPU for Server, low for Clients Server Client Input Extract & Synth 11

12 AMS Architectures (3) Distributed Processing Transmit extracted features Lower Bandwidth usage Lower CPU for Clients and Server Server Client Synthesize Input Extract Synthesize Synthesize Designing an AMS (3) Ex. Cambience Video Input Feature extraction on Server Transmission Scalar values Audio Sound synthesis on Client 12

13 Summary Abstract Media Spaces Throw away information Simple Degradation Feature Extraction Can provide a privacy shield Can provide better peripheral awareness Allow media re-mapping Can lower bandwidth usage Questions? 13

14 References 1) Pedersen, E. R., Sokoler,, T. (1997) AROMA: Abstract Representation of presence supporting Mutual Awareness. Proceedings of CHI 97, ) Wikipedia: : The Free Encyclopedia. (n.d( n.d.).) Retrieved October 2005 from en.wikipedia.org/wiki/abstract_art, en.wikipedia.org/wiki/pablo_picasso, 3) Smith, I., Hudson, S. (1995) Low Disturbance Audio For Awareness and Privacy in Media Space Applications. In proceedings of ACM Multimedia 95, ACM Press, p ) Boyle, M. (2005) Privacy in Media Spaces. PhD Thesis, Department of Computer Science, University of Calgary, Calgary, Alberta CANADA T2N 1N4. April. 14