Fast And Robust Interface Generation for Ubiquitous Applications

Transcription

1 Fast And Robust Interface Generation for Ubiquitous Applications The SUPPLE Project University of Washington, Seattle Krzysztof Gajos, David Christianson, Raphael Hoffmann, Tal Shaked, Kiera Henning, Jing Jing Long, and Daniel S. Weld

2 Automatic, On The Fly Fast And Robust Interface Generation for Ubiquitous Applications The SUPPLE Project University of Washington, Seattle Krzysztof Gajos, David Christianson, Raphael Hoffmann, Tal Shaked, Kiera Henning, Jing Jing Long, and Daniel S. Weld

3 Motivation

4 Motivation Variety of display devices & interaction contexts makes hand-designed interfaces expensive Adapt to device characteristics Current interfaces: complex & One size fits all Adapt to users and tasks Automatic interface generation is a scalable solution

5 Approach Develop abstract representation for: Interfaces Display devices Users Automatically generate interfaces from the abstractions

6 SUPPLE Architecture Interface Model SUPPLE User Model Application or Appliance Device Model User's Info Space Display Target Device

7 SUPPLE Architecture Interface Model SUPPLE User Model Application or Appliance Device Model User's Info Space Display Target Device

8 Road Map Motivation Modeling user interfaces in SUPPLE User interface generation as optimization Automatically adapting user interfaces A preliminary user study Adaptation in SUPPLE Customization support in SUPPLE Conclusions

9 Modeling User Interfaces simple types: int f loat str ing bool media types: image interactivemap containers: {τ i i 1...n } derivative types: τ, C τ vectors: vector(τ) actions: τ 1 τ 2

10 Modeling User Interfaces Classroom: τ: {,, } Light Bank: τ: {,, } A/V: τ: {, } Light: τ: {, } Light... Light... Projector: τ: {, } Light Level: τ: <int, [0,10]> Power: τ: bool Screen: τ: bool Power: τ : bool Input: τ: <string, {data1,data2, video}> Vent: τ: <int, [0,3]>

11 Modeling User Interfaces Classroom: τ: {,, } Light Bank: τ: {,, } A/V: τ: {, } Light: τ: {, } Light... Light... Projector: τ: {, } Light Level: τ: <int, [0,10]> Power: τ: bool Screen: τ: bool Power: τ : bool Input: τ: <string, {data1,data2, video}> Vent: τ: <int, [0,3]>

12 Modeling User Interfaces Classroom:!: {,, } Light Bank:!: {,, } A/V:!: {, } Light:!: {, } Light... Light... Projector:!: {, } Light Level:!: <int, [0,10]> Power:!: bool Screen:!: bool Power:! : bool Input:!: <string, {data1,data2, video}> Vent:!: <int, [0,3]>

13 Click! Model (UI, Device, User) - Algorithm - Results - Personalization (Adaptation, Customization) - Arnauld - Consistency - Task Models

14 Model (UI, Device, User) - Algorithm - Results - Personalization (Adaptation, Customization) - Arnauld - Consistency - Task Models

15 Click! Model (UI, Device, User) - Algorithm - Results - Personalization (Adaptation, Customization) - Arnauld - Consistency - Task Models

16 Model (UI, Device, User) - Algorithm - Results - Personalization (Adaptation, Customization) - Arnauld - Consistency - Task Models

17 Media Types

18 Click! Subtyping

19

20

21 Road Map Motivation Modeling user interfaces in SUPPLE User interface generation as optimization Automatically adapting user interfaces A preliminary user study Adaptation in SUPPLE Customization support in SUPPLE Conclusions

22 User Interface Generation as Optimization Driven by a Cost function ($) -- estimated user effort to manipulate a rendering of the interface Cost function derived from device model Algorithm: branch-and-bound search with full constraint propagation at each step Performance: seconds on a desktop computer Model (UI, Device, User) - Algorithm - Results - Personalization (Adaptation, Customization) - Arnauld - Consistency - Task Models

23 User Interface Generation as Optimization Flexible with respect to screen size Versatile: Same algorithm for different devices Allows new concerns to be included in the rendering process, e.g.: Cross-device consistency Adaptation to usage patterns

24 Robustly Adapting to Different Screen Sizes

25 Single Algorithm -- Many Devices

26 Adapting To Usage Patterns SUPPLE with an empty trace SUPPLE with a lights-heavy trace

27 Road Map Motivation Modeling user interfaces in SUPPLE User interface generation as optimization Automatically adapting user interfaces A preliminary user study Adaptation in SUPPLE Customization support in SUPPLE Conclusions

28 Adapting To Usage Patterns By Complete Makeover SUPPLE with an empty trace Is this the only way to adapt? SUPPLE with a lights-heavy trace

29 Content Eliding

30 Visual Popout

31 Split Interfaces

32 Adaptation Strategies Efficiency Complete Makeover Confusion

33 Adaptation Strategies Efficiency Complete Makeover Content eliding Confusion

34 Adaptation Strategies Efficiency Complete Makeover Content eliding Visual Popout Confusion

35 Adaptation Strategies Efficiency Complete Makeover Split Content eliding Visual Popout Confusion

36 Adaptation Strategies Efficiency Complete Makeover Split Content eliding Visual Popout Confusion

37 Preliminary User Study Split Interface Visual Popout Interface

38 Preliminary User Study Results Split Interface strongly preferred over non-adaptive Both adaptive interfaces slightly faster than non-adaptive But: Visual Popout interface often found distracting

39 Sneak Peak: Followup Study In a different setting, we tested three adaptation strategies including Split Interface and Visual Popout Interface Users strongly preferred and were significantly faster using Split Interface Users strongly disliked Visual Popout Interface with Mary Czerwinski and Desney Tan

40 Split Interfaces in SUPPLE Adapting by promoting hard-to-reach but frequently used functionality

41 Split Interfaces in SUPPLE Click!

42 Split Interfaces in SUPPLE Click! Click!

43 Split Interfaces in SUPPLE Click!

44 Split Interfaces in SUPPLE Four extra clicks required just to print in landscape mode!

45 Split Interfaces in SUPPLE

46 System- Vs. User-Initiated Adaptation: Customization Customizing any part of the interface with drag and drop Out of order undo Generalization

47 Customization

48 Customization

49 Customization

50 Customization

51 Customization

52 Road Map Motivation Modeling user interfaces in SUPPLE User interface generation as optimization Automatically adapting user interfaces A preliminary user study Adaptation in SUPPLE Customization support in SUPPLE Conclusions

53 Conclusions A powerful, flexible and practical tool for automatically generating user interfaces for multiple devices Adaptation and customization offer the possibility of creating custom-made UIs for each user Support for distributed operation and caching makes it practical even on small devices

54 Can I Have It? Yes! We are awaiting permission to release SUPPLE as an open source toolkit Visit SUPPLE web site to sign up for the user s mailing list

55 Contributors Daniel Weld Dave Christianson Raphael Hoffmann Tal Shaked Kiera Henning Jing Jing Long Anthony Wu me

56 More Information SUPPLE: : supple interfaces Krzysztof Gajos: kgajos@cs.washington.edu Daniel Weld: weld@cs.washington.edu

57 More Information SUPPLE: : supple interfaces Krzysztof Gajos: kgajos@cs.washington.edu Daniel Weld: weld@cs.washington.edu