A/B Testing + Why You Should Do It

Transcription

1 A/B Testing + Why You Should Do It

2 A/bout this webinar A/B testing overview: what and why? Approach to testing: behavioral framework Testing methodology: copy, visuals, layout A/B testing on website A/B testing s

3 What is A/B Testing? A B

4 Descendant of controlled experiment

5 Popularized by Google around 2000

6 What is A/B testing? A/B testing is comparing two or more versions of something to see which one performs better.

7 Why A/B Test? Everybody s doing it

8 Designs = best guesses Learn Build Measure

9 % lift in conversion 2.8m addresses $60m +/-

10 % lift in conversion 2.8m addresses $60m +/-

11 Netflix + chill

12

13

14

15

16 Benefits for Nonprofits Improve key engagement conversions ( signups, campaign participation, donations, etc) Improve site continuously Try before you buy

17 Defined Goals $ or s Sign-ups Desired/predictive Behavior Digital conversions tied to impact MVP modeling

18 Approach to Testing Theory & reality

19 What makes people do something? Passive Engaged

20 Behavior = motivation*ability*trigger Motivation: Undying love for food Ability: Abs + toned shoulders Trigger: Human command

21 Behavior = motivation*ability*trigger

22 A/bility - Can I do this easily? Give perception of ease Time: quick survey Effort: no credit card required Minimize number of taps

23 Goal: Donate Form Completion

24 Goal: Donate Form Completion +5%

25 Motivation - Do I want to do this? Focus on need fulfillment Social proof - others are using it Authority: 3rd Party Validation Scarcity: Leverage FOMO (missing out) Reciprocity: give something away

26 Goal: Join a Network Original (boring)

27 Dollar bet battle George vs. Ali

28 Dollar bet battle George +99% vs. Ali -32%

29 Winner

30 High intensity Angry Fearful Anxious Defensive Resentful Invigorated Confident Challenged Joyful Connected Negative (unpleasant) Positive (pleasant) Depressed Exhausted Burned out Hopeless Defeated Relaxed Mellow Peaceful Tranquil Serene Low intensity

31 Trigger - What do I click? Clear calls-to-action Ask at the right time Ask at different moments

32 Sticky CTA

33 Reality (data) check

34 Where to Test? So many pages, so little time

35 Types of Pages to Test Main landing pages Conversion (SEM) pages Pop-ups Page templates

36 What to Test? Look good, sound good, feel good

37 What to test: copy Body copy Headlines Calls-to-action Menu item name Donation amount

38 What to test: copy Body copy Headlines Calls-to-action Menu item name Donation amount

39

40 1.84% conversion rate 9.66% conversion rate

41 Improvement over time

42 Child Trends example

43 Child Trends example

44 What to test: Visuals Hero images Icons Colors

45 A Spice it up B Keep it simple

46 A Spice it up 4x teacher signups B Keep it simple

47 Audubon example

48 Audubon example

49 What to test: layout Order of elements Position of elements Element staticness Form length

50 A 2 CTAs B 1 CTA

51 A 2 CTAs 38.3% Clickthrough rate improvement B 1 CTA

52 America Recycles Day example Pledge form

53 America Recycles Day example Pledge form

54 Running experiments Ready, set, test

55 Some A/B testing tools

56 Build

57 Build

58 Measure

59 Measure

60 Learn

61 Where to start? What is the most important action users can take on your site? Which behavioral element (ability, motivation, trigger) can be improved? Which material element (copy, visuals, layout) can you test? What s your hypothesis? => Don t walk. Run the A/B test!

62 Newsletters Got your , now what?

63 Anatomy 101 Subject line Preheader Sender Preheader - small text that shows up in inbox list. Can help increase open rate!

64 A/B Testing

65 Preheader text and sender

66 Time (will tell) Test delivery time methods to optimize time of day: Send in batches Use Timewarp to cater to users time zone Send Time Optimization gives a recommendation based on zillions of data points

67 Subject line Test your subject line if you have more than 5,000 people. Which would you open? Typhoon Haiyan Support Slam What Would YOU Change?

68 Subject line Test your subject line if you have more than 5,000 people. Which would you open? Typhoon Haiyan Support Slam What Would YOU Change? 9,300 People difference for Power Poetry (List 100k)

69 Subject line length

70 Resources Optimizely Academy Whole Whale s 9 steps to run a/b test Mailchimp A/B test tutorial

71 Where to start? You know it matters, so GET STARTED! The only way to get it (completely) wrong is to not do it all.