FeedTree: Sharing Web micronews with peer-topeer event notification

Transcription

1 FeedTree: Sharing Web micronews with peer-topeer event notification Mike Helmick, M.S. Ph.D. Student - University of Cincinnati Computer Science & Engineering

2 The Paper: International Workshop on Peer-to-peer Systems - February 2005 Dan Sandler, Alan Mislove, Ansley Post, Peter Druschel Rice University - Department of Computer Science Some graphics are recreated from a presentation by Dan Sandler 2

3 News and the Web In the early days... Just a handful of infrequently updated news sites Sufficient to get updates by clicking reload or visiting again later 3

4 Situation today Uncountable proliferation of news sites micronews highly focused chunks of content, appearing frequently and irregularly, scattered across scores of sites. 4

5 RSS RDF Site Summary Rich Site Summary Really Simple Syndication XML Summary of the current content on the site 5

6 Blogs, etc. Just about everyone provides RSS feeds now News sites: CNN, NYTimes Blogs: LiveJournal, MovableType 6

7 RSS Distribution All clients poll a central server (usually the same as the web server) Bandwidth requirements are linear in the number of users (BAD!) 7

8 Solution... Distributed approach look at p2p overlays logarithmic scaling goal: reduce the load on the content providers and share the cost among all users 8

9 P2P Model 9

10 More on RSS XML based file format Often called a feed Many programs called aggregators or readers to process the feeds 10

11 RSS is meant to be consumed by? A People B Machines C D Dogs Reporters

12 RSS is meant to be consumed by? A People B Machines C D Dogs Reporters

13 Feeder for Mac OS X 13

14 Mozilla Firefox (Sage) 14

15 What s in the file? Headlines Complete stories Images (links to) Permanent URL (called a permalink) inbox for the internet 15

16 Bandwidth Considerations Sites that offer RSS tend to become more popular Out of balance: more RSS requests than interactive users 4 observed behaviors 16

17 Polling For each feed, for each user... Must issue HTTP requests over and over Site must respond to each requests - even if nothing has changed 17

18 Superfluity RSS data format is static Always returns the N most recent entries Not all of those may be new to the client (wasted bandwidth) 18

19 Stickiness Once someone subscribes to an RSS feed, they are likely to stick with that feed This creates long term bandwidth requirements for the server 19

20 24 hours traffic Web usage Typically rolling - users only hit web sites when awake RSS Usage Tends to be 24 hours a day on each users machine, compounding bandwidth requirements 20

21 Bandwidth Requirements slashdot.org 17,700 subscribers (according to Bloglines) Each RSS feed is 2kb Polled half-hourly 850,000 daily transfers 1.7 GB of data (per day) 21

22 What s been done? 22

23 Improved Polling Not practical Too hard to enforce restrictions on when clients can access Clients don t know when content changed or what changed 23

24 Outsorce Aggregation A central service dedicated to aggregation Built around a well-formed API But Danger of central control 24

25 FeedTree 25

26 Group Communication data distribution as an alternative to polling still does not scale well 26

27 Scribe Scalable group communication system Built on P2P overlay such as Pastry Each Scribe group has 160 bit group ID 27

28 Scribe / Pastry Pastry routes Scribe Tree 28

29 Additions to XML <rss version= 2.0 > <channel> <title>fooblog</title> <link> <scribe:topic> <scribe:pubkey>... </scribe:pubkey>... </channel> </rss> 29

30 More Scribe Comparable to IP Multicast link stress is less than double IP multicast Has self healing capabilities to detect broken links 30

31 What are the architectural Issues? A B C D

32 What are the architectural Issues? A Source of B Authenticity of Information Information C Delivery of D Verification of Information Information

33 What are the architectural Issues? A Source of B Authenticity of Information Information C Delivery of D Verification of Information Information

34 Source of Information RSS document must come from the source Each item is modified to aid clients in the detection of omitted or delayed events timestamp sequence number 34

35 What are the architectural Issues? A Source of B Authenticity of Information Information C Delivery of D Verification of Information Information

36 Source of Information RSS data is signed Using public key encryption 36

37 What are the architectural Issues? A Source of B Authenticity of Information Information C Delivery of D Verification of Information Information

38 Delivery of Information RSS document is multicast (using the overlay network) Peers who have subscribed to the topic (group ID) receive the feed 38

39 What are the architectural Issues? A Source of B Authenticity of Information Information C Delivery of D Verification of Information Information

40 Verification of Information Peers verify the signature of the feed The aggregator desktop application then notifies the user of new content 40

41 Additional Comments Client application bootstraps by making a conventional HTTP request for the RSS Additional updates from from FeedTree system Feeds are given a time-to-live with maximum interval between events Uses heatbeat events at the end of ttl to signal no change 41

42 Proliferation RSS has grown because of support publishers client applications 42

43 FeedTree Publishing CMS software takes part in overlay network long-lived node Publishing software transmits new messages when the site changes 43

44 FeedTree Reading RSS reading applicaitons join the FeedTree network as peers They now receive and distribute messages (routing) New events are presented to the user very quickly as they happen No changes are needed to client interface! 44

45 Benefits providers lower cost distribution avoidance of slashdotting users timelier updates 45

46 Recovery Remember the sequence numbers and timestamps? Clients can use these on updates or heartbeats to detect gaps Query parent for information Fallback to HTTP request if can t be found 46

47 Status of FeedTree Authors developed software prototype Have a server ftproxy that can handle publishing requested content According to one of the authors - as of Friday - this is still quite a work in process 47

48 What else can this be used for? 48

49 Questions? 49