Annoyed Users: Ads and Ad-Block Usage in the Wild Enric Pujol TU Berlin Oliver Hohlfeld RWTH Aachen Anja Feldmann TU Berlin IMC 15 Tokyo, Japan
2
http://www.journalism.org/2015/04/29/digital-news-revenue-fact-sheet Page Fair and Adobe report on Ad Blocking 3
Mozilla Add-on statistics 4
Publishers reaction The Guardian 5
(Why can I not see BILD.de?) 6
Acceptable ads program (non-intrusive ads) White-listing within AdBlockPlus (ABP) 7
Our work Residential Broadband Network Online advertising Methodology based on AdBlockPlus Ad-blocker usage Characterize ad traffic 8
9
10
URL Content-Type Base URL ads.example.com^ $ script, image, domain=bar.com DOM Tree #document HTML HEAD BODY IMG Content Policy www.bar.com www.foo.com http://ads.example. com/ad.gif Is this information available in HTTP header traces? 11
ads.example.com ^ $ script, image, domain=foo.com 12
ads.example.com ^ $ script, image, domain=foo.com HTTP logs Bro IDS 13
ads.example.com ^ $ script, image, domain=foo.com HTTP logs Bro IDS Map Web Pages ReferrerMap 14
ads.example.com ^ $ script, image, domain=foo.com HTTP logs Bro IDS Map Web Pages ReferrerMap Classify requests libadblockplus + filter lists 15
Limitations HTTP logs HTTPS traffic Content-Type Referrer Map Web page Classification Hidden ads (CSS) List sanity 16
Ad-blocker usage indicators Any ad-blocker AdBlock Plus Low ratio of ad requests Easylist updates Caveat: ad-free content Caveat: HTTPS + NATs How to find a suitable threshold? 17
Active measurements Browser Mode #HTTP ad-requests ratio Vanilla 57.8K 16 % Adblock Plus + all lists 48.5K <0.1 % Values for the complete set of Alexa top 1K 28.10.2015 18
When can we apply ad-ratio indicator? Vanilla: no plugin AdBP-Pa: adblockplus + all lists At least 1K requests. Set threshold to 5% 28.10.2015 19
Data sets Trace RBN-1 RBN-2 Duration 4 days 15 and ½ hours Subscribers 7.5 K 19.7 K HTTP bytes 18.8 TB 11.4 TB HTTP requests 131.95 M 85.09 M Characterize ad traffic Assess ad-blocker prevalence 28.10.2015 20
How much ad-related traffic? 0 % 18 % 100 % Ad-related traffic EasyList and derivatives (55 %) EasyPrivacy (35 %) Acceptable ads (10 %) Loose filters @@ gstatic.com^$document Black-listed (57%) 28.10.2015 21
How many ABP-capable browsers? 500K pairs <IP, UserAgent> 1.6K unique UserAgent strings 9.2 K users Heavy hitters Known browsers Requests: 50 % (all) 72 % (ads) 33 % 24 % 15 % 21 % 7 % Firefox Chrome Safari Mobile IE 28.10.2015 22
Ad Ratio EasyList Browsers with no evidence of an adblocker: 46% of the ad traffic 28.10.2015 23
Ad Ratio EasyList Inconsistent indicators a) various users in the same household 28.10.2015 24
Ad Ratio EasyList Inconsistent indicators a) different plugin, or b) ad-free content 28.10.2015 25
Ad Ratio EasyList Evidence of Adblock Plus for 22% of the users Still generate ad traffic!? 28.10.2015 26
How do users configure ABP? AGGREGATED 82% 11 % 7% EasyPrivacy Acceptable ads EasyList 99% EasyPrivacy 90% No ad blocker ABP users 87% Acceptable ads 70% Most AdBlockPlus users do not change default settings 27
Summary Can we use existing functionality to classify ad traffic? Yes, we can! (but limitations apply) 28.10.2015 28
Summary Can we use existing functionality to classify ad traffic? Yes, we can! (but limitations apply) Which traffic is potentially affected by Adblock Plus? ~18% requests and ~1% HTTP bytes Google, CDNs, Cloud Providers and dedicated ASes 10% of the ad-related traffic is white listed 28.10.2015 29
Summary Can we use existing functionality to classify ad traffic? Yes, we can! (but limitations apply) Which traffic is potentially affected by Adblock Plus? ~18% requests and ~1% HTTP bytes Google, CDNs, Cloud Providers and dedicated ASes 10% of the ad-related traffic conforms Acceptable Ads Can we measure how end users adopt ad-block plugins? Yes, in our traces ~20% of users Out-of-the-box i.e., acceptable ads / no EasyPrivacy 28.10.2015 30
Thank you! Can we use existing functionality to classify ad traffic? Yes, we can! (but limitations apply) Which traffic is potentially affected by Adblock Plus? ~18% requests and ~1% HTTP bytes Google, CDNs, Cloud Providers and dedicated ASes 10% of the ad-related traffic conforms Acceptable Ads Can we measure how end users adopt ad-block plugins? Yes, in our traces ~20% of users Out-of-the-box i.e., acceptable ads / no EasyPrivacy 28.10.2015 31
28.10.2015 32