UI Toolkits. HCID 520 User Interface Software & Technology

Transcription

1 UI Toolkits HCID 520 User Interface Software & Technology

2

3 Xerox Alto 1973

4 Evolution of User Interfaces Command Line (UNIX shell, DOS prompt) Interaction driven by system User prompted for input when needed Text-based input and output Event-Driven Interfaces (GUIs) Interaction driven by user System waits for user action and responds Pointing & text input, graphical output More complex architecture & development

5 Component Model

6

7

8

9

10 Component (Widget) Model Encapsulate interactive components (widgets) Component library (button, slider, container) Interface built as a hierarchy of components Widgets drawn by underlying graphics library Input event generation and dispatch Historically mouse & keyboard, now touch, Bounds management & damage/redraw Model geometry, redraw updated regions only

11 User Interface Components Each component (widget) is an object with: A bounding box A method for drawing itself Drawn in the component s coordinate system Methods (callbacks) to process input events public void paint(graphics g) { g.fillrect( ); // interior g.drawstring( ); // label g.drawrect( ); // outline }

12 User Interface Components Label TextArea Each component has its own 2D coordinate system. Buttons

13 Containment Hierarchy Window Panel Label TextArea Panel Button Button

14 Component Layout Each container is responsible for positioning its contents. Window Panel Label TextArea Panel Button Button

15 Event-Driven Architecture

16 Events User input is modeled as events. Events are dispatched to components for processing.

17 Events User input is modeled as events. Events are dispatched to components for processing. Hardware Events Mouse: move, button-down, button-up Keyboard: key-down, key-up

18 Events User input is modeled as events. Events are dispatched to components for processing. Hardware Events Mouse: move, button-down, button-up Keyboard: key-down, key-up Inferred Events Mouse: click, double-click, enter, exit, drag Keyboard: key-press Window: move, resize

19 Anatomy of an Event Events encapsulate the information needed for components to react to input or changes. Event Type (mouse moved, key down, etc) Event Target (the input component) Timestamp (when did event occur) Modifiers (Ctrl, Shift, Alt, etc) Event Content: Mouse: x,y coordinates, button pressed, # clicks Keyboard: which key was pressed

20 Demo: Browser DOM Events Go to Open the JavaScript console, enter: var h = document.queryselector("header"); h.addeventlistener("click", function(event) { console.log(event); }); Click the header region at the top of the page. Inspect the events in the JavaScript console.

21

22 Event Dispatch Loop Event Queue List of input events Mouse moved (t 0,x,y) Event Loop (runs in dedicated thread) Remove next event from queue Determine event type Dispatch to proper component(s) Invoke callbacks on components Continue, or wait until event arrives Component Invoked callback method Update application state Request repaint, if needed

23 Event Dispatch Loop

24 Event Dispatch Loop Mouse moved (t 0,x,y)

25 Event Dispatch Loop Event Queue List of input events Mouse moved (t 0,x,y)

26 Event Dispatch Loop Event Queue List of input events Mouse moved (t 0,x,y) Event Loop (runs in dedicated thread) Remove next event from queue Determine event type Dispatch to proper component(s) Invoke callbacks on components Continue, or wait until event arrives

27 Event Dispatch Loop Event Queue List of input events Mouse moved (t 0,x,y) Event Loop (runs in dedicated thread) Remove next event from queue Determine event type Dispatch to proper component(s) Invoke callbacks on components Continue, or wait until event arrives Component Invoked callback method Update application state Request repaint, if needed

28 Event Dispatch Window Event Queue Mouse moved (t 0,x,y) Mouse pressed (t 1,x,y,1) Mouse dragged (t 2,x,y,1) Key typed (t 3, F1 ) Panel Label TextArea Panel Button Button

29 Event Dispatch Window Event Queue Mouse moved (t 0,x,y) Mouse pressed (t 1,x,y,1) Mouse dragged (t 2,x,y,1) Key typed (t 3, F1 ) Panel Label TextArea Panel Button Button

30 Event Dispatch Window Event Queue Mouse moved (t 0,x,y) Mouse pressed (t 1,x,y,1) Mouse dragged (t 2,x,y,1) Key typed (t 3, F1 ) Panel Label TextArea Panel Button Button

31 Event Dispatch Window Event Queue Mouse moved (t 0,x,y) Mouse pressed (t 1,x,y,1) Mouse dragged (t 2,x,y,1) Key typed (t 3, F1 ) Panel Label TextArea Panel Button Button

32 Event Dispatch Window Event Queue Mouse moved (t 0,x,y) Mouse pressed (t 1,x,y,1) Mouse dragged (t 2,x,y,1) Key typed (t 3, F1 ) Panel Label TextArea Panel Button Button

33 Event Dispatch Event Queue Mouse moved (t 0,x,y) Mouse pressed (t 1,x,y,1) Window Mouse dragged (t 2,x,y,1) Key typed (t 3, F1 ) Panel Label TextArea Panel /* callback for TextArea */ function mousemoved(evt) { // process mouse moved event } Button Button

34 UI Tools Discussion

35 Myers, Hudson & Pausch Brief UI tool history, establish vocabulary in wide use today, and chart future trajectories. At any point did they miss the mark?

36 Myers, Hudson & Pausch Brief UI tool history, establish vocabulary in wide use today, and chart future trajectories. At any point did they miss the mark? Web: subsuming many desktop applications Design vs. Dev: In practice often done by teams with varied skills (interaction design, graphic design, implementation) 3D: Mostly limited to games? BUT fabrication now likely to drive new 3D design tools. VR?

37 Toolkits Abound Not just for desktop GUI widgets & events Mobile applications (ios, Android) Information visualization (d3.js, Protovis, Prefuse) Tangible user interfaces (Papier-Maché) Gestural user interfaces (Kinect API) Speech user interfaces (MS Speech API) Proxemic user interfaces (Proximity Toolkit) Many exhibit similar tensions around expressiveness, efficiency and ease-of-use

38 Toolkit Benefits & Shortcomings Simplify & accelerate UI development Lower the threshold of specification difficulty Raise the ceiling of what can be expressed But hard to do both! Establish path of least resistance Makes common / successful designs easy Makes unusual / ineffective designs harder Can limit & shape what we conceive / design Enforce consistency of design Maintain familiar widgets and interactions

39 EXERCISE 1 Widget Design

40 Hello

41 Default State Hello

42 Default State Data: styling, content, etc. Hello

43 Default State Data: styling, content, etc. Hello form-checkboxes-and-radio-buttonssky-ui-kit-psd/ FID=9&TID=10550 Origin Unknown.

44 Checked State Hello FID=9&TID=10550 Origin Unknown.

45 Disabled-Unchecked Disabled-Checked Focus Maybe-Checked HTML/Forms/The_native_form_widgets FID=9&TID=10550 Origin Unknown. library/ui-elements/html5- checkboxes

46

47 State Diagram to the Rescue Click Checked Disabled Default Click

48 State Diagram to the Rescue Click Checked Disabled Default Click Click Maybe

49 State Diagram to the Rescue Click Checked Disabled Default Click Shift- Click Click Maybe

50 Your Turn! Submit! In pairs, brainstorm the states + transitions for a submit button.! What does the button look like in each state?! What are the transitions between states?! What actions trigger the transitions?! How does a user know what to do?! Regroup in ~10 mins to share ideas.

51 Your Turn! I am a textbox! In pairs, brainstorm the states + transitions for a textbox.! Hint: States+transitions for the textbox but also its contents (e.g., text selection).! Regroup in ~10 mins to share ideas.

52 Your Turn: Interactive Prototypes