This talk is about split testing

Transcription

1 @dom_woodman

2 This talk is about split testing

3 Money.

4 Talked ourselves out of money How to get the money back

5 Talked ourselves out of money How to get the money back

6 How did we talk ourselves out of the money?

7 Part 1

8 Not knowing things is scary

9

10

11 ?

12

13

14

15 We re paid experts, of course we should know

16 Part 2 Fear 3

17 SEO is hard to measure

18 Before

19 Before After

20

21 Not money. At least we re on sessions

22 Still not money. Now we re on clicks.

23 What on earth is search visibility?

24 Seasonality

25 Seasonality Industry Demand

26 Seasonality Industry Demand Google

27 Seasonality Industry Demand Google

28 Ranks

29 Ranks CTR estimates

30 Ranks CTR estimates Handwaving

31 No-one cares when the going is good

32 Measurement Fear Part 3

33 Part 3: SEO has become more individual

34 More ranking factors Which interact in more complex ways

35 Measurement Fear Complexity

36 So how does this all fit together

37 So how does this all fit together

38 So how does this all fit together

39 So how does this all fit together

40 So how does this all fit together

41 So how does this all fit together

42 So how does this all fit together

43 So how does this all fit together

44 So how does this all fit together

45 So how does this all fit together

46 So how does this all fit together

47 So how does this all fit together

48 So how does this all fit together

49 So how does this all fit together

50 So how does this all fit together

51 So how does this all fit together

52 So how does this all fit together

53 So how does this all fit together

54 What does my CV look like? SEO Manager: Jan Jan 2017 SEO Manager at T-Shirts and Shorts. Completed 4 major projects. Reduced Dev backlog. Removed a penguin penalty. Regularly audited backlinks. Organic traffic went up from 100K ~ 150K

55 What does my CV look like? -- If I m unlucky SEO Manager: Jan Jan 2017 SEO Manager at T-Shirts and Shorts. Completed 4 major projects. Reduced Dev backlog. Removed a penguin penalty. Regularly audited backlinks.

56 Talked ourselves out of money How to get the money back

57 Stop making guesses about our results

58 CRO and growth hacking are doing it right

59

60

61 Learning from split tests

62 What does an SEO split test look like? Control Variant

63 This is a traditional CRO test Control Variant

64 I m about to show you 2 graphs a lot. This is the first.

65 Black is the variant, treat blue as the control

66 In this case the blue line is higher, it s a negative test

67 This is the start date

68 The second graph shows the total additional sessions

69 The fans show margin of error

70 The test is significant when the fan crosses 0

71 In this case it happened here:

72 Here they are together.

73 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data

74 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data

75 Established wisdom would say ALT tags on images are good for SEO

76 We found adding ALT tags to images had no effect

77 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data

78 How does SEO text on category pages perform?

79 E-commerce site number 1 - No effect/negative effect

80 E-commerce site number 2 ~ 3.1% increase in organic sessions

81 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data

82 We can render Javascript!

83 What happens if your content is only visible with Javascript? Javascript Disabled Javascript Enabled

84 Making it visible increased organic sessions by ~ 6.2%

85 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data

86 What happens when you match title tags to the greatest search volume? Title tag before: Which TV should I buy? - Argos Title tag after: Which TV to buy? - Argos

87 Organic sessions decreased by an average of 8%

88 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data

89 Category pages have lots of images and not much text

90 Adding structured data to category pages

91 Organic sessions increased by 11%

92 Running your own tests

93 1.How much traffic do you need? 2.How long should you run a test for 3.What kind of things are easy to test? 4.How can you do all the maths?

94 1.How much traffic do you need? 2.How long should you run a test for 3.What kind of things are easy to test? 4.How can you do all the maths?

95 Average of 1,000 organic sessions per day to your set of pages

96 1.How much traffic do you need? 2.How long should you run a test for 3.What kind of things are easy to test? 4.How can you do all the maths?

97 Typically takes 2-3 weeks to reach significance.

98 1.How much traffic do you need? 2.How long should you run a test for 3.What kind of things are easy to test? 4.How can you do all the maths?

99 Anything where you re making changes to a page

100 1.How much traffic do you need? 2.How long should you run a test for 3.What kind of things are easy to test? 4.How can you do all the maths?

101 distilled.net/splittester

102 bit.ly/testing-lessons

103