Essential Tools For Finding And Fixing Customer Experience Problems

Transcription

1 Essential Tools For Finding And Fixing Customer Experience Problems Bill Albert, PhD Executive Director, Design and Usability Center Bentley University June 26, usability.bentley.edu

2 Agenda Setting the stage Review of UX tools When to use these tools Takeaways 2

3 Setting the stage

4 About Me Executive Director, Design and Usability Center, Bentley University I am data driven! I love tools to evaluate and measure the customer experience I create connections between UX research and practice 4

5 Multi Channel Experience What is the true customer experience? How do I gain insight into the customer experience? How do I measure it? What do I do with this information? 5

6 Process Plan Identify Type of Tool Select Tool(s) Use Tool(s) Question 6

7 Framework Attitudes Behaviors What are people doing? Why is it a problem and how to fix it? What is the problem and how big is it? What are people saying? Qualitative Quantitative 7

8 Common UX Tools

9 Common UX Tools Attitudes Behaviors Usability Lab Testing (formative) Ethnographic Observation Diary Studies Unmoderated Usability Web Analytics A/B Testing Eye Tracking Physiological Click/Mouse Contextual Inquiry/In-Depth Interviews Focus Groups Online Surveys Card Sorting/IA VOC Data Mining Qualitative Quantitative 9

10 Common UX Tools Attitudes Behaviors Usability Testing (formative) Ethnographic Observation Diary Studies Unmoderated Usability Web Analytics A/B Testing Eye Tracking Physiological Click/Mouse Contextual Inquiry/In- Depth Interviews Focus Groups Online Surveys Card Sorting/IA VOC Data Mining Qualitative Quantitative 10

11 Usability Testing (formative) Goal is to identify significant usability issues with a design Focus on usage, less on preferences Works with any interface Observing users creates a powerful experience 11

12 Usability Testing: Video Tools Quick and dirty qualitative feedback Web-based Easily identify most significant pain points No opportunities to probe Create videos, pull verbatim comments 12

13 Ethnographic Observation Observation of product usage in natural environment Validate designs in the real world Greater customer insight into actual product usage Define user requirements Sparks innovation 13

14 Common UX Tools Attitudes Behaviors Usability Testing (formative) Ethnographic Observation Diary Studies Unmoderated Usability Web Analytics A/B Testing Eye Tracking Physiological Click/Mouse Contextual Inquiry/In- Depth Interviews Focus Groups Online Surveys Card Sorting/IA VOC Data Mining Qualitative Quantitative 14

15 Contextual Inquiry/In-Depth Interviews Structured interviews in natural context Probing questions about usage, goals, motivations, task flows, and requirements Develop personas Insight into barriers to entry, unmeet needs, and opportunities for design improvement 15

16 Focus Groups Explore product/design concepts with a group of targeted customers or prospects Additional activities might include sketching, card sorting, or pairedcomparisons Compare design alternatives, early prototypes, and wire frames Insight into early design concepts, user needs, and attitudes/preferences 16

17 Attitudes Behaviors Common UX Tools Usability Testing (formative) Ethnographic Observation Diary Studies Unmoderated Usability Web Analytics A/B Testing Eye Tracking Physiological Click/Mouse Contextual Inquiry/In- Depth Interviews Focus Groups Online Surveys Card Sorting/IA VOC Data Mining Qualitative Quantitative 17

18 A/B Testing/Multivariate Testing Divert web traffic to an alternate design(s), and compare against existing design (live site) Typically subtle design changes Increase conversions, decrease abandonment through series of tests Challenge is to learn, and propagate success! Source: 18

19 Eye Tracking Eye tracking technology collects data about where participants are looking in real time Answers questions such as: How much time is spent looking at different objects? What do participants notice first? What regions of the display receive the most attention? How long does it take a participant to notice an object? Test with static images, websites, print media, video, or commercials 19

20 Eye Tracking 20

21 Physiological Measures emotional engagement based on skin conductance Incorporates facial expressions to measure valence (positive/negative emotions) Useful for products/experiences that intend to elicit emotion Best when comparing designs and experiences Great for use in the field, and in association with eye tracking 21

22 Physiological 22

23 Click and Mouse Captures data about where people click and move their mouse, and scrolling behavior Insight into what elements draws attention, and actual usage 23

24 Common UX Tools Attitudes Behaviors Usability Testing (formative) Ethnographic Observation Diary Studies Unmoderated Usability Web Analytics A/B Testing Eye Tracking GSR/Physiological Click/Mouse Contextual Inquiry/In- Depth Interviews Focus Groups Online Surveys Card Sorting/IA VOC Data Mining Qualitative Quantitative 24

25 Online Surveys Mostly quantitative feedback about attitudes, opinions, and preferences Collect qualitative feedback about experience Some tools measure clicks (hot spots and heat maps) Paints a more accurate picture of the experience, with statistical reliability 25

26 Card Sorting / IA Participants organize items/content into categories and may provide labels (open sort) Alternatively, participants may place items in a pre-existing set of categories (closed sort) Insight into how users organize content/functionality, and labeling Validate proposed or current information architectures 26

27 Common UX Tools Attitudes Behaviors Usability Testing (formative) Ethnographic Observation Diary Studies Unmoderated Usability Web Analytics A/B Testing Eye Tracking GSR/Physiological Click/Mouse Contextual inquiry In-depth interviews Focus Groups Online Surveys Card Sorting/IA VOC Data Mining Qualitative Quantitative 27

28 Unmoderated Usability Self-guided online survey based on actual behavior Collect qualitative feedback about experience Collect quantitative data about usage such as task success, time, abandonment, and clicks Paints a more accurate picture of the experience, with statistical reliability 28

29 When to use UX tools

30 When to Use UX Tools Discovery Surveys Ethnographic Observation Diary Studies Focus Groups Usability Testing (Baseline/Competitive) Card sorting (open) Contextual Inquiry/IDI VOC/Data Mining Design Design/Evaluation & Benchmarking Validation Concept Development Early Structural Design Detailed and Visual Design Production Release 30

31 When to Use UX Tools Discovery Design Design/Evaluation & Benchmarking Usability Testing (formative) Low/High Fidelity Competitive Card Sorting (open/closed) Surveys Click/Mouse Unmoderated Usability Validation Concept Development Early Structural Design Detailed and Visual Design Production Release 31

32 When to Use UX Tools Discovery Design Design/Evaluation & Benchmarking Validation Unmoderated Usability (summative) Surveys A/B Testing/Multivariate Testing Web Analytics VOC/ Data Mining Click/Mouse Card Sorting (closed) Concept Development Early Structural Design Detailed and Visual Design Production Release 32

33 Takeaways

34 My Five Awards 1. Most Overlooked: Card Sorting 2. Richest Dataset: Unmoderated Usability 3. Biggest Bang for the Buck: Usability Testing (lab or remote) 4. Greatest Insight: Ethnographic Observation 5. Best Long-Term Investment: A/B Testing and Customer Panels 34

35 Keys to Success Clearly define the question(s) you want to answer Challenge your assumptions Don t boil the ocean Approach data skeptically Triangulate Try before you buy 35

36 Questions? Bill Albert, PhD Executive Director Design and Usability Center,