Understanding Your Audience: Using Probabilistic Data Aggregation Jason Carey Software Engineer, Data

Transcription

1

2 Understanding Your Audience: Using Probabilistic Data Aggregation Jason Carey Software Engineer, Data

3 The Vision

4 Insights Audience API Fast, ad hoc aggregate queries 10+ proprietary demographic models Device, Gender, Interest, Language, Location, TV Audience size up to tens of millions of users User privacy is of paramount importance

5 Birdseye View User: a Twitter user People tweeting about doughnuts People tweeting : a collection of users used to build audiences Audience: a collection of segments that can be queried Audience Audience Query: an audience and a collection of demographics Query Interest by Gender Audience Aggregate Results

6 Example Query For people tweeting about my brand, tell me the top TV shows by gender: { } Female : { Game of Thrones : 7, House : 24, The Daily Show : 4, Wimbledon : 14 }, Male : { Game of Thrones : 21, House : 9, The Daily Show : 18, Wimbledon : 16, } 7% of audience are females following Game of Thrones

7 Technical Approach

8 Technical Approach Users in a

9 Technical Approach Users in a Users in a Random Sample

10 Technical Approach When adding users to a segment, maintain a random customer add users to segment API

11 Technical Approach When adding users to a segment, maintain a random customer add users to segment API When creating an audience, merge segment s into an audience customer create audience API segment s audience

12 Technical Approach When adding users to a segment, maintain a random customer add users to segment API When creating an audience, merge segment s into an audience customer create audience API segment s audience When querying, call demographic index for users in audience and aggregate results customer query audience API audience demographic index

13 Sampling Technical Challenges customer can be updated over multiple requests Duplicate users may be sent in requests create segment add users customer API store segment add users API is sent in add update API update

14 Sampling Maintain a MinHash for each segment hash(@katy) Add 4 users to segment hash(@bob) API customer 9 7 MinHash Sample size: 2 hash(@bob) 7 (@jeff, 4) hash(@jeff) 4 MinHash is the random Store MinHash

15 Audience Sampling Technical Challenges Duplicate users across segments customer create audience API segment s audience Merging segment s into a single random 10,000 Users Doughnut 1,000,000 Audience

16 De-Duping Users In Samples If the same user exists in multiple segment s, does it matter how you de-dupe? 10,000 Users Doughnut 1,000,000 Audience

17 De-Duping Users In Samples If the same user exists in multiple segment s, does it matter how you de-dupe? No, it does not matter as long as the audience is constructed using a weighted selection based on the segment cardinalities. 10,000 Users Audience 1,000,000 Users

18 De-Duping Users In Samples 10,000 Users 1,000,000 Users Audience with higher probability with lower probability

19 De-Duping Users In Samples Option in Doughnut Option in Coffee 10,000 Users 1,000,000 Users 10,000 Users 1,000,000 Users Doughnut Coffee Audience Audience with higher probability with lower probability

20 De-Duping Users In Samples NEED TO KNOW THIS 10,000 Users 1,000,000 Users Doughnut Coffee HyperLogLog to the rescue! Audience with higher probability with lower probability

21 Did Somebody Say HyperLogLog? Estimates the number of unique items in a data stream A billion items requires only KBs of storage for an accuracy of 2% Intuition: I flip a coin a bunch of times and tell you the largest number of heads flipped in a row. I then ask you how many times I flipped the coin. Flajolet, P.; Fusy, E.; Gandouet, O.; Meunier, F. (2007). "HyperLogLog: the analysis of a near-optimal cardinality estimation algorithm" (PDF). AOFA 07: Proceedings of the 2007 International Conference on the Analysis of Algorithms.

22 Merging Samples Repeat until size is reached Weighted selection of segment based on segment s cardinality 10,000 Users Doughnut Sample 1,000,000 Users Coffee Sample Randomly choose a user from selected segment User 7 User 1 User 4 User 2 Audience Sample User 9

23 Sampling Recap MinHash for sampling users in a segment HyperLogLog for estimating segment cardinality s de-duped in any order Weighted selection of segment s for audience

24 User Privacy

25 Don t Forget About User Privacy Make it extremely difficult to reverse engineer a user s demographics Attack vectors Set balancing Side channel Homogenous group search

26 Set Balancing (Simple) Query 1 Query 2 Users 1 though 20 Users 1 though 21 Is information leaked?

27 Set Balancing (Simple) Users 1-20 Is information leaked?

28 Set Balancing (Simple) User 21 Is information leaked? Yes! Q2 - Q1 leaks info about User 21

29 Set Balancing (Complex) Execute a series of queries Query 1 Users 1 though 10 Query 2 Query 3 Query 4 User 1 plus users 11 through 20 Odd users between 1 and 20 Even users between 1 and 20 Is information leaked?

30 Set Balancing (Complex) Execute a series of queries Query 1 Users 1 though 10 Query 2 Query 3 Query 4 User 1 plus users 11 through 20 Odd users between 1 and 20 Even users between 1 and 20 Is information leaked?

31 Set Balancing (Complex) Execute a series of queries Query 1 + Query 2 Query 3 + Query 4 Is information leaked?

32 Set Balancing (Complex) Execute a series of queries User 1 Is information leaked? Yes! Q1 + Q2 - Q3 - Q4 leaks info about User 1

33 Mitigating Attacks Minimum audience size Random sampling Reporting thresholds Query auditing

34 Architectural Highlights Valuable insights while protecting user privacy Small storage footprint Query latency does not increase with audience size

35 Thank