An Overview of Search Engine Spam. Zoltán Gyöngyi

Transcription

1 An Overview of Search Engine Spam Zoltán Gyöngyi

2 Roadmap What is search engine spam? What techniques do spammers use? Work at Stanford Challenges ahead Stanford Security Workshop Stanford, March 19,

3 Example kaiser pharmacy online Stanford Security Workshop Stanford, March 19,

4 Example mp3 Save today on Viagra, Lipitor, Zoloft, Phentermine 90 Pills/$119 Stanford Security Workshop Stanford, March 19,

5 Example Lawyers Loans Mortgage Ringtones Viagra Pharmacy is the profession of compounding and dispensing medication. More recently, the term has come to include other services Stanford Security Workshop Stanford, March 19,

6 Definition So who does what? Stanford Security Workshop Stanford, March 19,

7 Definition So who does what? Spamming deliberate human action Stanford Security Workshop Stanford, March 19,

8 Definition So who does what? Spamming deliberate human action meant to trigger unjustifiably high ranking Stanford Security Workshop Stanford, March 19,

9 Monetization Why? Better ranking = higher click-through rate Search engine optimization Affiliate spam Advertisement spam Stanford Security Workshop Stanford, March 19,

10 Techniques How? Stanford Security Workshop Stanford, March 19,

11 Techniques / Boosting / Term how? boosting techniques hiding link Stanford Security Workshop Stanford, March 19,

12 Techniques / Boosting / Term repetition repetition repetition repetition repetition repetition dumortierite dumose dumous dump dumpage dumper dumpily dumpiness dumping dumpish dumpishly work in weaving three-women teams is an ancient textile art on looms please refrain from using the phrase stitching wounds located on the lower limbs Stanford Security Workshop Stanford, March 19,

13 Techniques / Boosting / Link boosting term techniques hiding Stanford Security Workshop Stanford, March 19,

14 Techniques / Boosting / Link Directory clones Duplicate (parts of) DMOZ or Yahoo! Directory Comment spam Post messages (containing links) to Blogs (Unmoderated) forums Wikis Link spam farms Create colluding spam pages See later Stanford Security Workshop Stanford, March 19,

15 Techniques / Hiding term techniques boosting term how? Stanford Security Workshop Stanford, March 19,

16 Techniques / Hiding Content hiding <style type = text/css > body { background-color: white; color: white; } </style> <a href = ><img src = 1x1.gif ></img></a> <div style = visibility: hidden >You can t see me!</div> Cloaking Identify web crawlers Serve a different version of the page Stanford Security Workshop Stanford, March 19,

17 Roadmap What is search engine spam? What techniques do spammers use? Work at Stanford Challenges ahead Stanford Security Workshop Stanford, March 19,

18 Work at Stanford Analysis Link spam farms and alliances Demotion TrustRank Detection Spam mass estimation See publications Stanford Security Workshop Stanford, March 19,

19 Link Spam Farms & Alliances Spammer s goal: increase PageRank Farm model k 0 k Stanford Security Workshop Stanford, March 19,

20 Link Spam Farms & Alliances Optimal farms Short loops including target Stanford Security Workshop Stanford, March 19,

21 Link Spam Farms & Alliances Optimal farms Short loops including target Alliances Interconnected farms 2 always better than 1 Larger alliances often benefit all Stanford Security Workshop Stanford, March 19,

22 TrustRank / Observation good pages spam pages Stanford Security Workshop Stanford, March 19,

23 TrustRank / Observation good pages spam pages Stanford Security Workshop Stanford, March 19,

24 TrustRank / Observation good pages spam pages Stanford Security Workshop Stanford, March 19,

25 TrustRank / Observation good pages spam pages Approximate isolation of good pages: good pages seldom point to spam Stanford Security Workshop Stanford, March 19,

26 TrustRank / Objective Separate good pages from spam pages Stanford Security Workshop Stanford, March 19,

27 TrustRank / Objective Separate good pages from spam pages What? Assign high scores to very good pages How? Propagate scores from known good pages (seed set) When? Use results in ranking Stanford Security Workshop Stanford, March 19,

28 TrustRank / Example Stanford Security Workshop Stanford, March 19,

29 TrustRank / Example Stanford Security Workshop Stanford, March 19,

30 TrustRank / Example Stanford Security Workshop Stanford, March 19,

31 TrustRank / Example 1.50 Damping Stanford Security Workshop Stanford, March 19,

32 TrustRank / Example Splitting Stanford Security Workshop Stanford, March 19,

33 TrustRank / Experiments Data Site-level AltaVista web graph: 31M sites Seed set of 178 sites Evaluation sample 1000 manually tagged sites Results Log scores 20 buckets Top 5 PageRank buckets: 15-20% spam Top 5 TrustRank buckets: almost no spam Stanford Security Workshop Stanford, March 19,

34 Roadmap What is search engine spam? What techniques do spammers use? Work at Stanford Challenges ahead Stanford Security Workshop Stanford, March 19,

35 Challenges Remove economic incentive Why not just charge for high ranking? Revenue based on transactions generated, not click-through rate Mechanism design Spam-proof algorithms/services Spam on community-driven sites Flickr, MySpace, del.icio.us Stanford Security Workshop Stanford, March 19,

36 Thank You! Stanford InfoLab Publications Contact Stanford Security Workshop Stanford, March 19,

37 TrustRank / Experiments Web data Entire AltaVista index (June 2003) Site-level web graph 31M nodes 13M without inlinks Seed set 2500 candidates 178 selected high-quality sites Evaluation sample 1000 manually tagged sites Stanford Security Workshop Stanford, March 19,

38 TrustRank / Experiments Stanford Security Workshop Stanford, March 19,

39 TrustRank / Experiments Stanford Security Workshop Stanford, March 19,

40 TrustRank / Experiments Average Demotion (# of Buckets) Spam from PageRank bucket #3 moved to TrustRank bucket # PageRank Bucket Stanford Security Workshop Stanford, March 19,