CS415 Human Computer Interaction

Transcription

1 CS415 Human Computer Interaction Lecture 3 WIMP HCI (Part 1 - GUI Builder Concepts) September 10, 2017 Sam Siewert

2 Consider Is the text from page 21 below correct? What is the value of color vision to humans at 1/20 th resolution of grayscale? (6 million cones, 120 million rods) Sam Siewert 2

3 Reading and Notes Summary P. 21 the is repeated twice ( over the the lazy dog.) Automatic correction based on expectation Parsing line by line What causes humans to miss what should be obvious in visual scenes? [page 21 of text] Adaptation of cues useful in nature used differently for HCI Limit and focus on what s important (visual), not all details are beneficial Normalization (habitual) and seeing what is expected (lowers cognitive load) not noticing a change (missing objects) Brain forms an image filling in details (that might not be there) (cognitive model) What is the value of color vision to humans about 1/20 th resolution of grayscale? (4.5 to 6 million cones, million rods) Information fruit ripe yet?, dangers such as poisonous animal Same shape, but different color (distinguish), draw focus Essential shades (could be grayscale) RED means stop or angry (implied semantics), lighter colors more peaceful. To distinguish objects (shape, texture, shading). Osterberg, Gustav. "Topography of the layer of rods and cones in the human retina." Acta ophthalmol 13 (1935): Sam Siewert 3

4 A Little Humor on HCI Sam Siewert 4

5 WIMP GUI Builders The 2D GUI (e.g. , word processors, spreadsheets) is the Killer App for WIMP Floating Window in Main Desktop Window Integrates Various Widgets Window Canvas [Background] Buttons (Radio Selection, Single-Click-Icons, Toggles) Text Entry and Display Pop-Up and Pull-Down Menus Sliders and Scrolling (for large windows) Minimization and dismissal Handles Desktop Integration Hide and Expose Events Pointer Selection Events Look and Feel of Existing Windows Sam Siewert 5

6 WIMP Windows Icons Menus Pointers State of Practice Introduced in 1980 s!! Apple Macintosh Desktop GTK+, QT, Java, Python, and Numerous More GUI Builders Quality and Efficiency Improvements (Multi-programming and Multi-threading) Sam Siewert 6

7 Multi-Programming Multi-Programming Invented for SLOW Storage Devices (Namely Disk and Tape) Waiting on Disk and Other I/O Where p=fraction of time waiting on I/O, n=number of processes, CPU utility = 1 p n Degree of multiprogramming Modern Operating Systems (3rd Edition) 3rd Edition Sam Siewert by Andrew S. Tanenbaum 7

8 Disk Arm Motion Random IOPs are the Problem (200 per Disk Typical) Due to Seek time + ½ Rotation on Average (milliseconds) An Eternity compared to Nanoseconds (1, 10, 100 s) for CPU and Memory running at GhZ Rates Sam Siewert 8

9 Disk Access Latency Calculations Sam Siewert 9

10 Multi-Threading Lighter Weight Concurrency for Multi-core, Load Balancing, AMP and SMP (Asymetric and Symmetric Multi-Processing) For Servers (Many Outstanding Service Requests) For Desktops ( , Edit, Backup, Simulate,, all in Different Windows at the same time ) Thread Grids Divide and Conquer All Embarrassingly Parallel Problems Sam Siewert 10

11 Many Core System on a Chip Cluster on a Chip (E.g. Intel MICA Many Integrated Core Architecture) Real-Time Rendering Graphics Rendering and Inverse Rendering with Computer Vision Sam Siewert 11

12 GUI Layout & Code-Gen Tools or VHLLs for GUI Building Method #1 Compiled Programming Language (e.g. C/C++) and Widget Libraries (e.g. X-windows Gnome, KDE, etc. or Windows) Advantages Compiled for Efficiency, Security, Proprietary IP Protection (Stripped Binaries) Disadvantages Less Interactive Development: Layout, Generate code, Compile, Test, Repeat Method #2 Interpreted Very High Level Language (e.g. Java, Python, C# (both with Common Language Infrastructure)) Advantages Interpreted for Quick Test as you Code (Agile), Rapid Prototyping, Portability Disadvantages Security Risks in Runtime (Java Virtual Machine, CLI) Sam Siewert 12

13 Elements of a GUI Builder Layout Tool with Code Generation API for Simple Layout Main Window and Realization of it in Main program Canvas (Background) Layout Widget (where to place sub-windows and widgets inside application main window) Active and Passive widgets Active widget event handlers Filling in Event Handling Behavior State machine or State transition design Sam Siewert 13

14 State Machine vs. State Transition Table State Machine Shows Design Intent - User Enters 1,2,3 Select Start FSM = 20% Coverage [Finite State Machine] State Transition Table Shows all Possible Transitions E.g. States S={Start, A, B, C, End} E.g. Input I={select, 1, 2, 3, Enter} Table = S x I 25 Combinations of S with I A C 1 2 B 3 Enter End Moore FSM Output => State Next => Current + Input State / Input Select Enter Start A???? A? B??? B?? C?? C??? End? End???? Start Mealy FSM Output => State + Input Next state => Current + Input Sam Siewert 14

15 Assignment #2 Dr. Java + Swing Qt Creator for C++ Visual Studio WinForm C# Xamarin C# for Mobile Apps Python PyCharm Java Example JDK Dev Kit JRE Runtime (libs, plugins, etc.) JVM Virtual Machine (Runs programs) Somewhat Complex Sam Siewert 15

16 Java OOP Java Calculator Example Main program container class CalculatorGui class Public and Private classes and methods Sam Siewert 16

17 Java Calculator Walkthrough and Demo Import of Java Swing GUI Widget set Instantiation of Calculator GUI Main window configuration and realization import javax.swing.*; import java.awt.*; import java.awt.event.*; Main widget construction public class SwingCalculator { public static void main(string[] args) { CalculatorGui cg = new CalculatorGui(); Main widget configuration } } cg.setsize(250,400); cg.setresizable(false); cg.setdefaultcloseoperation(jframe.exit_on_close); cg.setvisible(true); Main widget realization Sam Siewert 17

18 Advantages Java for WIMP GUIs Portable runs on Java Virtual Machine Interactive development with IDE (DrJava or Eclipse) Well Designed OOP Widely Used Lots of Tutorials and Documentation Good for mobile systems (e.g. Android OS) Disadvantages Interpreted Byte Codes or Just-in-Time Compilation (Slower than Native Compiled code) Requires installation of JVM (Java Virtual Machine) Distribution requires JAR (Java Archvie) Security Concerns Oracle Resources Sam Siewert 18

19 import javax.swing.*; import java.awt.*; import java.awt.event.*; Deeper Dive public class CalcTemplateGui extends JFrame { JFrame frame = new JFrame("My Simple Calculator"); JTextField answerfield; JButton add, sub, div, mul, equals; JButton one, two, three, four, five, six, seven, eight, nine, zero; JPanel contentpanel; String stemp1=""; // CalculatorGui default constructor - super long constructor public CalcTemplateGui() { super("my Simple Calculator"); } private class Operations implements ActionListener { public void actionperformed(actionevent event) { JButton src = (JButton) event.getsource(); } } private class ArgumentEnter implements ActionListener { public void actionperformed(actionevent event) { JButton src = (JButton) event.getsource(); Event handler Classes for specific Active widgets 0-9 Digits and Arithmetic Operators } } } Sam Siewert 19

20 Connecting Buttons to Event Handlers Instantiate Handler Classes for Button Types Add Action Listener Using Swing Implement Handler Class and Detect Source // layout with specific dimension for each button one.setpreferredsize(dim); two.setpreferredsize(dim); div.setpreferredsize(dim); mul.setpreferredsize(dim); equals.setpreferredsize(dim); // create a private class to handle events detected by an action listener Operations op = new Operations(); ArgumentEnter arg = new ArgumentEnter(); // Add a listener to each button to invoke a handler for arguments entered one.addactionlistener(arg); two.addactionlistener(arg); nine.addactionlistener(arg); zero.addactionlistener(arg); // Add a different listener to each button to invoke a handler add.addactionlistener(op); sub.addactionlistener(op); div.addactionlistener(op); mul.addactionlistener(op); equals.addactionlistener(op); Sam Siewert 20

21 GUI Builder & Code Challenges Builders that Generate Code can get Out of Sync with Event Handlers Generates Separate Handler Files (Classes) Should not Overwrite Handler Logic Structure Logic in External Modular Code GUI Code Becomes Redundant Similar Code for Each Button (Active Widget) Many buttons Similar Code for Layout Similar Code for Panels, Menus, etc., differing only by Labels and Use OOP Can Abstract and Allow for Instances that Inherit and Refine Sam Siewert 21

22 Interactive Features - Handlers Implements Tasking and State Machine Logic of GUI Function of Current Button Depends on History of Previous Button Events Menus are Context Sensitive (Some Options Grayed Out) Pop-Ups Generated Based on Task Stream and Errors E.g. Are you Sure - Confirm Deletion of File E.g. Exit without Saving? Display Updates Require History Cancel Undo/Redo Context Sensitive Help Pop-Ups Sam Siewert 22

23 Event Handlers Ideally Simple State Machines May Require Concurrency Display Updates (Telemetry, Monitors) Pointer Drive Selections (Buttons, Menus, Sliders, Scrolling) Client/Server Interaction Status Updates [Progress] Seems Simple, but Challenging Due to Multiplicity of Simplicity Verification and Validation Use as Intended, Stress Test (Unexpected Use) Sam Siewert 23

24 GUI Layers GUI Application Build on API or CLI Simply Generates CLI Commands or API Calls Updates Displays with Return Codes or Screen Scraped Messaging Not Always the Best Idea (but Often Done) Sam Siewert 24

25 Prototypes and Mock-ups Early Feedback on Look and Feel and Interaction Usability Engineering Time to Complete Common Tasks Success Rate for Test Tasks by Different Users Cognitive Load for Tasks using an HCI or GUI Focus Groups for Usability Sam Siewert 25