C ISPA. Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification. Ben Stock 29th Annual FIRST Conference

Size: px

Start display at page:

Download "C ISPA. Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification. Ben Stock 29th Annual FIRST Conference"

Adam Bridges
6 years ago
Views:

1 Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification Ben Stock 29th Annual FIRST Conference

Our work: understand how notifications can work at scale What are suitable communication channels for such a campaign?

2 Motivation of our Work Large-Scale (Web) Vulnerability Detection Drupaggedon SQLi, Joomla! Object Deserialization, Client-Side XSS, Execute After Redirect on Ruby, Focus previously on Detection, not Notification Our work: understand how notifications can work at scale What are suitable communication channels for such a campaign? Does such a campaign affect the prevalence of the notified vulnerabilities? What might inhibiting factors be? Today s talk: get insights from the CERT community 2

3 Study Setup

Types of Vulnerabilities Well-known vulnerabilities for WordPress (43,865 domains, Top 1M) Reflected Cross-Site Scripting in PlUpload flash component (CVE-2013-0237) Client-Side Cross-Site Scripting

4 Types of Vulnerabilities Well-known vulnerabilities for WordPress (43,865 domains, Top 1M) Reflected Cross-Site Scripting in PlUpload flash component (CVE ) Client-Side Cross-Site Scripting in Genericons Example Code (CVE ) XMLRPC Multicall Vulnerability allows attacker to try multiple user/password combinations in a single HTTP request Existing patches for all of them Previously-unknown Client-Side XSS vulnerabilities (925 domains, Top 10K) Site-specific flaws No existing patches 4

Communication Channels Direct Communication Channels Web contact forms Generic email addresses (info@, security@, webmaster@, abuse@) Domain WHOIS information (registrant or technical contact)

5 Communication Channels Direct Communication Channels Web contact forms Generic addresses Domain WHOIS information (registrant or technical contact) Indirect Communication Channels Vulnerability Reward Programs Hosting providers (abuse contacts for the hosting IP range) Trusted Third-Parties regional CERTs (e.g., CERT US, CERT-Bund) FIRST trusted community Ops-Trust 5

Notification Procedure Split up data set of vulnerable domains into five groups of equal size Generic, WHOIS, Provider, TTP, and Control Notification via email with link

6 Notification Procedure Split up data set of vulnerable domains into five groups of equal size Generic, WHOIS, Provider, TTP, and Control Notification via with link to our Web interface alternatively: access via using token Aggregated Disclosure to providers and TTPs Bi-weekly s January 14th, January 28th, February 11th 6

7 Web Interface 7

Reachability Analysis Mailbox and accessed reports to classify domains reached: report viewed or email acknowledged bounced: all emails for this domain

9 Reachability Analysis Mailbox and accessed reports to classify domains reached: report viewed or acknowledged bounced: all s for this domain bounced unreachable: no WHOIS contact, no provider abuse mail, or redirect to Web interface unknown: all others indirect channels: first step of the chain measured 9

10 Global Impact of Notification

11 Fixed Sites over Time WordPress +280 (+3.1%) domains Client-Side XSS 25% Generic Domain Cont. Provider 25% Generic WHOIS Provider 31 (16.8%) TTP Control TTP Control domains 20% 20% fraction of fixed domains 15% 10% fraction of fixed domains 15% 10% 5% 5% 0% 0% 01/14 01/21 01/28 02/04 02/11 01/14 01/21 01/28 02/04 02/11 11

12 Although the notifications for both WordPress and Client-Side XSS showed significant improvements over the control group, the number of domains which were fixed is unsatisfactory (25.8% and 12.6%, respectively).

13 Communication Channel Analysis

14 Reachability of Direct Channels Bounced Unreachable Unknown Reached Viewed 100 % 80 % 60 % 40 % 20 % 0 % Generic WordPress Generic Client-Side XSS WHOIS WordPress WHOIS Client-Side XSS 14

15 Reachability of Direct Channels Bounced Unreachable Unknown Reached Viewed 100 % 80 % 60 % 40 % 550 WHOIS domains report reads, but only 280 fixed 20 % 0 % Generic WordPress Generic Client-Side XSS WHOIS WordPress WHOIS Client-Side XSS 15

16 Reachability of Indirect Channels Bounced Unreachable Unknown Reached Viewed 100 % 80 % 60 % 40 % 20 % 0 % Provider WordPress Provider Client-Side XSS TTPs WordPress TTPs Client-Side XSS 16

17 Reachability of Indirect Channels Bounced Unreachable Unknown Reached Viewed 100 % 80 % 60 % 40 % 70 TTP domains report reads, but only 31 fixed 20 % 0 % Provider WordPress Provider Client-Side XSS TTPs WordPress TTPs Client-Side XSS 17

18 Time to Fix after Report View Same rate as the 50% 50% control group after % fixed domains after view 40% 30% 20% 10% day 5. Generic WHOIS Provider 0% 0days 5days 10 days 15 days 20 days TTP % fixed domains after view 40% 30% 20% 10% 0% 0days 5days 10 days 15 days 20 days WordPress Client-Side XSS 18

19 Key Insights and Follow-Up Questions

20 Establishing Communication Channels Direct channels are hard to reach generic s perform really bad for average Web sites WHOIS helps, but is incomplete (~18.5% without entry) Indirect channels are easier to reach Often do not forward the information top 5 providers (~25% of domains) did not react How can the security community come up with reliable means of establishing communication channels between researchers and affected parties? 20

21 Need for Reminders and Time to Fix Reminders helped especially for direct channels Generic WHOIS Provider TTP Once report was viewed, fix ratio was ~25-30% after five days, WordPress fix rate equaled control group viewed domains Future notification campaigns should make 0 01/17 01/21 01/25 01/29 02/02 02/06 02/10 02/14 WordPress frequent use of reminders How can we improve on the fix ratio? viewed domains Generic WHOIS Provider TTP /17 01/21 01/25 01/29 02/02 02/06 02/10 02/14 Client-Side XSS 21

more inclined to act upon German CERT info What is the impact of the sender

22 Sender Reputation Previous work found that sender reputation does not matter Our work begs to differ German CERT more inclined to forward information Providers more inclined to act upon German CERT info What is the impact of the sender reputation, especially when using intermediaries, on the success of a notification campaign? 22

User Distrust Our experiments required users to click a link or send an email with a token Community trains users not to click/react Notified control group with full disclosure email results only

23 User Distrust Our experiments required users to click a link or send an with a token Community trains users not to click/react Notified control group with full disclosure results only differed significantly for WordPress BUT: performed worse than with links! Potential issue in the message length To what extent does the message tone, content, and length influence the success of notification campaigns? 23

Results Generality Results appear to be dependent on the domain Providers worked best for Network vulns and Heartbleed Even within the same domain, results differ e.g. Generic on WordPress v.

24 Results Generality Results appear to be dependent on the domain Providers worked best for Network vulns and Heartbleed Even within the same domain, results differ e.g. Generic on WordPress v. Client-Side XSS Are campaigns more successful if the vulnerabilities gained attention in the media (such as Heartbleed)? Does it matter who needs to fix the vulnerability, be it a network admin, Web site developer, or enduser? 24

25 Connecting with the FIRST Community Is it feasible to notify WordPress at scale? Should we use different formats, endpoints,..? How could we make the reports more useful? What else can the research community do to ensure that vulnerability notifications can work at scale? 25

26 Conclusion Conducted first analysis into notifications for Web vulnerabilities at scale two data sets: well-known (WordPress) and previously-unknown (Client-Side XSS) flaws four communication channels: direct (generic s, WHOIS) and indirect (providers, TTPs) Results show statistically significant improvement caused by our campaign WHOIS worked best for WordPress, TTP best for Client-Side XSS Overall improvement was unsatisfactory 74.5% of all domains in data set vulnerable at the end of our experiments Main problem is reaching administrators in the first place 30% fix rate within five days (WordPress) / 25% (Client-Side XSS) Thank you! Questions? 26

C ISPA. Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification

C ISPA. Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification Ben Stock, Giancarlo Pellegrino, Christian Rossow, Martin Johns, and Michael Backes USENIX Security Symposium Austin,