Securing Behavior-based Opinion Spam Detection

Transcription

1 Securing Behavior-based Opinion Spam Detection Shuaijun Ge, Guiiang Ma, Sihong Xie and Philip S. Yu Dec 13,

2 Fake reviews? 2

3 Online reviews Source: based on a pool of representative sample of 1,031 US-based consumers 3

4 The challenges Source: based on a pool of representative sample of 1,031 US-based consumers 4

5 Eisting efforts Outcome Eplanations ReviewMeta.com 1. Feature engineering 2. Detection models Help make decision 5

6 Detection ratio of SR number of reviews rating Time series motif finding Graphical models Language models Detection outcomes time Features Time series Graphs Tets Evade Data Pollute 6

7 Spamming Account Detection Deviation from avg g Linear model g Spammers Normal users Attack gradient False positive Number of 5-star posts per day 7

8 Accessing knowledge of detector (publications) What yelp fake review filter might be doing, ICWSM, 2013 Deviation from avg Number of 5-star posts per day ma # of review per day Positive ratio Review length Rating deviation Maimum content 8 similarity

9 Accessing knowledge of detector (Detection websites) Deviation from avg 0.7 Number of 5-star posts per day 9

10 Accessing knowledge of detector (Released data) Deviation from avg g Number of 5-star posts per day 10

11 To defend: need to generate the attacks. Deviation from avg g Linear model Actionable? Attack parameters: # of 5-star per day = 4 Dev from avg = 0.5 Number of 5-star posts per day 11

12 To defend: need to generate the attacks. How? Actionable attack 3 post 4 per day post 1 per day post 1 per day Actionable attack 1 post 4 per day post 1 per week Actionable attack 2 post 3 per day Actionable? Attack parameters: # of 5-star per day = 4 Dev from avg =

13 Spammer objective function = (risk of being detected) (profit of spamming) Temporal anomalies AVG rating Change in rating Deviation from predicted avg Predicted AVG rating 13

14 Spammer objective function = (risk of being detected) (profit of spamming) Rating distribution anomaly : Background : Rating dist at time t 14

15 Spammer objective function = (risk of being detected) (profit of spamming) Rating distribution anomaly : Rating dist at time t : Rating dist at time t1 15

16 Spammer maimizes [risk of being detected profit of spamming] 16

17 Find amout of promotion Manipulated AVG rating Cap of all ratings <= 5 Deviation from predicted avg Predicted promotion >= 0 promotion Temporal change in AVG Organic AVG are set to 80 th percentiles of the corresponding changes estimated from the historic data 17

18 Find a proper amount of promotion in AVG rating Manipulated Large temporal change in AVG? Organic AVG are set to 80 th percentiles of the corresponding changes estimated from the historic data 18

19 find a proper number of spamming ratings Manipulated NR Large incremental in the number of reviews? <= 80 th percentile of historic increments Organic NR Number of spams Large absolute number of reviews? <= 80 th percentile of historic NR 19

20 Compute an evasive rating distribution : Background : Rating dist at time t from the last step Optimal rating distribution found by the dual problem. 20

21 The found evasive rating distribution

22 Fleible attacks generation. Evade time series based detectors E1: NR E2: NR NR E3: NR deviation in AVG rating E4: NR deviation in AVG rating change in AVG rating Evade both time series and rating distribution based detectors E5: KL-DIV E6: KL-DIV change in entropy E9: Ma Entropy change in entropy For short history targets E-A: NR NR change in AVG rating Ma Entropy E-B: NR NR change in AVG rating 22

23 Targets with long review histories Products with >= 1,000 reviews Reviews span more than 37 months (Yelp) / weeks (Amazon) 1,175,088 reviews / 383 products 247,117 reviews / 327 restaurants. Targets with short review histories The remaining products / restaurants are used. Longitudinal data are too sparse for each target. Probe parameters Attack! Probe parameters Attack! Product 1 Long-history data last 5 weeks short-history last 5 weeks Probe parameters Attack! short-history last 5 weeks Product 383 Long-history data last 5 weeks 23

24 Average spams posted by each attack 24

25 Attacking rate (% of windows can be spammed) 25

26 Promotion in ranking per spam 26

27 Secure the detector again Deviation from avg g Linear model Re-trained linear model Spammers Normal users False positive Number of 5-star posts per day g Attack gradient 27

28 Probe parameters Attack simulation Attack in the wild! Target First 30 weeks last 5 weeks Model re-training Pooling (DETER) Training data generated from E1 Training data generated from E2 model 1 model 2 Ensemble Training data generated from E9 model 9 28

29 Full information detection / evasion game: single spammer Detector Game 1 Game 2 KL-DIV # of reviews KL-DIV Detector # of reviews Spammer E1 E Spammer E1 E equilibrium

30 W^m W^a W^r EN_A EN_M DETER Ma-min Ma of signals Avg of signals Randomly selection Re-train avg Re-train Ma Re-train Pool Game equilibrium 30

31 W^m W^a W^r EN_A EN_M DETER Ma-min Ma of signals Avg of signals Randomly selection Re-train avg Re-train Ma Re-train Pool Game equilibrium 31

32 Unsupervised Attack agnostic Simple and good performance Good for long and short review histories Can secure the detector! Source codes and data avaiable at:

33 Thank you