Due Today: Painting App

Transcription

1 Evaluation

2 Due Today: Painting App Implement a very simple Java painting application supporting the following functions: Draw curves, filled rectangles and filled ovals specified by a mouse drag (don't worry about dynamically drawing the shape during the drag - just draw the final shape indicated). Shape selection (line, rectangle or oval) selected via combo box OR menu. Color selection using radio buttons OR menu. Line thickness using a combo box OR menu. A CLEAR button. Whatever else you want to throw in

3 Due Next Class: P4 Design Sketches Explore possible design options, and sketch what interfaces will look like. Interaction Scenarios Expand each activity design scenario (3+) into full interaction scenarios, thinking about what the user perceives and the actions he/she performs at each major step in the scenario. Design Options Three options for most important window/dialog box Brief rationale for why you selected one over the other two. Preliminary interface design. One or more sketched windows or dialog boxes, along with the menus and controls that the user manipulates.

4 Evaluation Why evaluate? What needs to be evaluated? How do we evaluate?

5 Why Evaluate To measure something (duh) e.g., usability metrics (learnability etc) To compare metrics to stated criteria To compare metrics from two or more alternatives To identify specific design problems that need to be fixed

6 UI Evaluation Methods (subset) Expert/Inspection methods Heuristic evaluation Cognitive walk-through Modeling User Testing Qualitative methods (interviews, questionnaires, think aloud) ethnography; focus groups Quantitative methods Descriptive studies Experiments Qualitative vs. Quantitative

7 Expert Inspection Methods Heuristic evaluation Checklist approach Independent evaluators (3-5) Severity rating for problems (Nielson) 0. No problem 1. Cosmetic problem 2. Minor low priority 3. Major problem high priority 4. Catastrophe must fix

8 Heuristic Evaluation Problems Poor agreement among experts Evaluating your own designs: Difficult to ignore knowing how the system works Give the design the benefit of the doubt Find obscure flaws which few users will encounter. Woolrych and Cockton (2000) conducted a large-scale trial of heuristic evaluation. Expert issues were not experienced by people, Some severe difficulties were missed Conclusion: for formative testing only

9 Expert Inspection Methods Cognitive walkthrough Walk through each step in the task and evaluate: i. Is the effect of the action the same as the user s goal at that point? ii. Will users see that the action is available? iii. Once users have found the action, will they know it is the one they need? iv. After the action is taken, will users understand the feedback they get?

10 Model-based Evaluation Build and evaluate a formal model e.g. GOMS (Goals, Operators, Menus, Selection)

11 User Testing Who? Setting? Pros / Cons?

12 User Testing Methods Qualitative Think aloud Focus groups Cooperative evaluation Co-discovery Usability test = quantitative measures +think aloud + interview Living labs Controlled experiments Quantitative

13 Some Terminology Formative vs. Summative Evaluation Validity Ecological Internal External Significant Between vs. Within Subjects Design

14 Vredenburg et al Benyan, Table 10.2, pg 225 Pros/Cons of Expert vs. User testing?

15 Scenario 1 Mary has just designed a web site for her daughter s girl scout troup, allowing the public to order cookies. She d like the site to be as usable as possible. She has no budget, and no access to users, but does have a copy of Nielsen s usability book. What is the most appropriate evaluation method for her situation? Why?

16 Scenario 2 Megabuck Inc s R&D department has just designed a new replacement for the desktop mouse that they say will revolutionize computing by cutting time-to-target in half. Prove it says the CEO. What is the most appropriate evaluation method? Why?

17 Scenario 3 Startup Industries is thinking of developing a new web portal linking office gossip blogs, and have developed an early prototype, but they re not sure if anyone will want to use it. What kind of evaluation method should they use? Why?

18 One Key Factor to point out... There is no perfect experiment Methods each have strengths and weaknesses. You cannot compensate for all experimental factors You can ALWAYS have more subjects. Easier to be critical than positive Outside of experimental settings: correlation!= causation

19 Evaluation Strategy What Am I Evaluating? (prototype or?) What Constraints Do I Have? Money Time Availability of usability equipment Availability of participants and the costs of recruiting them Availability of evaluators Documenting the Evaluation Strategy

20 Other Ways of Measuring Usability... Auto logging data A / B testing Quantiative behavioral measure Eye gaze Biometric information Number of swear words per minute

21

22 Experimental Research Plans Experiments can take a huge amount of time to plan and prepare. Extreme example: medical clinical trials What kinds of things do you need to worry about when planning a study?

23 Research Statement A one to two sentence summary statement precisely describing the phenomena being investigated and how it will be examined. Research Question: Precise statement of the question driving the experimental approach Hypothesis: The expected outcome from an experiment.

24 Sample Research Question Why are social networking sites harmful? Better: How are online users experiencing or addressing privacy issues on such social networking sites as Facebook and Twitter? Is there a significant difference in mental rotation trial times between men and women after exposure to VR Shepard-Metzler motor training?

25 Sample Research Plan: The Effect on Head Calibration on Motor Performance in VR Research Plan (Hypotheses) Background Research / Previous Work Preliminary Studies (Pilots) Research Design and Methods Human Subjects

26 Experimental Approach 24 subjects (13 female) ages Performed a series of 4 x 12 Fitts-like reciprocal tapping tasks using a magnetic tracker and VR registered wooden board

27 Results

28 Exercise What would be an appropriate research question for this experiment?

29 Types of Quantitative Studies Descriptive Correlational Surveys & Mechanical Turk Studies Experimental Between-subjects Single factor, two-level Single factor, N-level (for N>2) Two factor, N-level (for N>=2) Within-subjects Single factor, two-level

30 Exercise Project teams Sketch two test plans Evaluate design alternative for interface sketches (P4) Evaluate your FINAL project

31 Swing Layout Manager Get ready to swear

32 JFrame A stand alone window...but what if you want sub-components?

33 JFrame's Layers (We will only focus on the content pane)

34 Sample Code class MyFrame extends JFrame { public MyFrame() { super("my Example"); setdefaultcloseoperation(jframe.exit_on_close); //populate the content pane: getcontentpane() pack(); } public static void main(string args[]) { java.awt.eventqueue.invokelater(new Runnable() { public void run() { new MyFrame().setVisible(true); } }); } }

35 JDialog Just like a JFrame except you can make it modal Note: Use JOptionPane for simple, standard alert & informational message dialogs. JColorChooser, JFileChooser built in, special-purpose dialogs.

36 Layout Managers Decide how to display the Components within a Container. To use a layout manager: Construct an instance of the manager. Assign the instance to the container using: setlayout(layoutmanager) Each Container can only have one layout manager. Or in NetBeans: R-click on component and choose Set Layout

37 FlowLayout The default for JPanel Strategy: Keeps components at their preferred size. Place components in rows, left-to-right. When a row fills up, a new row is started. Rows can be centered, left or right justified.

38 FlowLayout Example class FlowLayoutExample extends JFrame { public FlowLayoutExample() { super("flow Layout Example"); setdefaultcloseoperation(jframe.exit_on_close); Container frame=getcontentpane(); frame.setlayout(new FlowLayout(FlowLayout.LEFT,10,10)); frame.add(new Button("Button 1")); frame.add(new Button("Button 2")); frame.add(new Button("Button 3")); frame.add(new Button("Button 4")); frame.add(new Button("Button 5")); pack(); } }

39 BorderLayout Partitions layout into regions Assign to specific regions Regions scale differently

40 BorderLayout Code class BorderLayoutExample extends JFrame { public BorderLayoutExample() { super("border Layout Example"); setdefaultcloseoperation(jframe.exit_on_close); Container frame=getcontentpane(); frame.setlayout(new BorderLayout()); frame.add(new Button("Button 1"),BorderLayout.NORTH); frame.add(new Button("Button 2"),BorderLayout.SOUTH); frame.add(new Button("Button 4"),BorderLayout.WEST); frame.add(new Button("Button 5"),BorderLayout.CENTER); pack(); } }

41 GridLayout Forms a rectangular grid You specify the number of rows and columns All cells are the same dimensions Grid is filled left-to-right, topdown Constructor: GridLayout(int rows,int cols) 0 = undefined

42 GridLayout Example class GridLayoutExample extends JFrame { public GridLayoutExample() { super("grid Layout Example"); setdefaultcloseoperation(jframe.exit_on_close); Container frame=getcontentpane(); frame.setlayout(new GridLayout(0,2)); frame.add(new Button("Button 1")); frame.add(new Button("Button 2")); frame.add(new Button("Button 3")); pack(); } }

43 CardLayout Swaps among each of its components Each component can be named: add( name,component) First component displayed initially To swap among components CardLayout.next(Container parent) CardLayout.first(Container parent) CardLayout.last(Container parent) CardLayout.show(Container parent, name )

44 JTabbedPane Acts like a JPanel with a CardLayout

45 Exercise: How would you create this layout?? <see blackboard>

46 Homework I6: Layout Managers Your objective in this assignment is to get some experience with Frames, Dialogs and layout managers in Swing. Create your own (ideally project-related) application with the following minimum requirements: A JFrame and a (non-modal) JDialog. A JTabbedPane and JScrollPane. Nested JPanels including the following layout managers: GridLayout, FlowLayout, BorderLayout

47 I6 Requirements (cont.) Some interaction widgets (JButton, etc.) on every JPanel and tab (unless it makes no sense) Reasonable behavior when the JFrame is resized. (minimum size is allowed) Remember: design then code. Simpler layout manager hierarchies are much easier to debug. NOTE: You may not use GridBagLayout, Free Design, Box, Overlay, Null or Absolute Layout anywhere in the project. Please run other layouts by me.

48 To Do Read Rettig article on paper prototyping 3 Research articles on paper prototyping (critique) Finish by next class P4 design sketches C3 - Paper prototyping research papers. Finish by end of spring break I6 Swing Layout Managers