PubSubHubbub Real-time RSS for the Decentralized Web. Brett Slatkin Google Inc. 18 November 2009

Size: px

Start display at page:

Download "PubSubHubbub Real-time RSS for the Decentralized Web. Brett Slatkin Google Inc. 18 November 2009"

Posy Ryan
6 years ago
Views:

1 PubSubHubbub Real-time RSS for the Decentralized Web Brett Slatkin Google Inc. 18 November 2009

2 Motivation

3 Motivation Want server-to-server, interoperable messaging - Decentralized social networks - Federated messages ( ++) - Instant syndication - Data for mashups

4 Motivation Requirements - Easy to deploy - Topic-oriented - High-throughput - Enables new use-cases Not trying to solve client/server

5 Motivation Many protocols out there - Improved polling -- changes.xml, weblogs.com, Ping-o-Matic, rsscloud, SUP - Publish/Subscribe -- Jabber/XMPP pubsub, AMQP, RestMS, WS-Notification,... Good ideas, but - Not the right tool for topic-based messaging... or - More complex than most people need... or - Hard or unfamiliar to deploy... or - Not scalable in the long term

6 Motivation Goal compared to Wave - Wave is a reinvention of collaboration on the web - PubSubHubbub bootstraps the web for real-time Meet in the middle

7 Introduction

8 Polling Publisher Subscriber

9 Polling Publisher Subscriber Feed

10 Polling Please give me your feed. Publisher Subscriber Feed

11 Polling Please give me your feed. Publisher Subscriber Feed

12 Polling No!!! Are we there yet!? Publisher Subscriber

13 ... Publisher Hub Subscriber

14 PubSubHubbub Publisher Hub Subscriber

15 PubSubHubbub Publisher Hub Subscriber Feed Please give me your feed. I use this hub for real-time delivery

16 PubSubHubbub Subscribe me to Are the feed you at this sure? Yes! URL Publisher Hub Subscriber

17 PubSubHubbub Please My feed give at this me URL your has updated feed Publisher Hub Subscriber Add new content Feed

18 PubSubHubbub Find what s new Here s new content! Publisher Feed Hub Subscriber Okay!

19 PubSubHubbub Publisher Hub

20 PubSubHubbub Here s new content, everyone! Publisher Hub Sub Sub Sub Sub Sub Sub Sub Sub Sub Sub

21 Demo

22 PubSubHubbub benefits Bootstraps existing feeds to be real-time Simple for publishers and subscribers Hub provides reliability, scalability Easy security model

23 Scale

24 Scaling work with feeds How much does a single feed update cost? Very important as update frequency increases! Total Cost = Frequency x Marginal Cost

25 Scenario: Managing a library Collection of books for borrowing Need to tell the central office which books were borrowed

26 Doing work late War & Peace A Tale of Two Cities Don Quixote Romeo & Juliet.-~.~.~.-~.~.~.-~.~.~ 1984

27 Doing work late Date Title Borrower 11/9 Tale of two... Bob War & Peace Romeo & Juliet.-~.~.~.-~.~.~.-~.~.~ 11/10 Don Quixote Marsha 11/ Sally

28 Doing work late

29 Doing work late

30 Doing work late Date Title Borrower

31 Doing work late Date Title Borrower 11/9 Walden Sally 11/10 Cat in the Hat Bob /12 Huck Finn Coco

32 Doing work late Problem: Generating a list of borrowers is slow - Need to do full inventory each time the list asked for - Incremental cost is high Duh! Who would ever run a library this way?

33 Doing work late with feeds Today with polling and ping/polling - Must completely regenerate the feed on each update - Similar to our library Problems - Database load producing the feed - Bandwidth of the full feed document (to all subscribers) - Invalidation of cached feeds

34 Doing work late with feeds Total Cost = Frequency x Marginal Cost

35 Doing work early Date Title Borrower

36 Doing work early Date Title Borrower 11/9 Walden Sally 11/10 Cat in the Hat Bob /12 Huck Finn Coco

37 Doing work early Date Title Borrower 11/9 Walden Sally 11/10 Cat in the Hat Bob /12 Huck Finn Coco 11/18 War & Peace John

38 Doing work early Bookkeeping saves time - Before: Linearly increasing cost - After: Constant marginal cost Incremental, immediate picture of book status

39 Doing work early with feeds Publisher Publisher/ Hub combined Hub

40 Doing work early with feeds Hub pushes content directly - Only generate the piece of the feed that has changed - Do all work at write time! Benefits - No extra database load (beyond update cost) - Bandwidth of the partial feed document - No need for cache invalidation

41 Doing work early with feeds Total Cost = Frequency x Marginal Cost

42 Scale: Conclusion PubSubHubbub provides the best of both - Delegate to one or more Hubs to be real-time today - Run your own hub for peer-to-peer, high-throughput A migration path for new use-cases!

43 Progress

44 Progress Over 100 million feeds are PubSubHubbub enabled Companies - Google, FriendFeed (Facebook), livedoor, Six Apart, LiveJournal, Posterous, Pheedo, TwitterFeed, Superfeedr, LazyFeed,...

45 Progress Open-source libraries - Perl, PHP, Python, Ruby, Java, Haskell, C#, MovableType, WordPress, Django, Zend, Drupal,... Open-source hubs - App Engine, Erlang, Python, Ruby, Perl,...

46 Progress Will be licensed under Open Web Foundation - No Patents, Royalty-free, etc

47 Getting involved Active mailing list, 250+ members - Join! Review the specification yourself - Open process for feedback and peer review Write some sample code; contribute to open source Write on your blog why we need push for the future

48 Conclusion

49 Conclusion Make your feeds real-time with PubSubHubbub Enable new use-cases in the future Push it to the limit!

50 More info pubsubhubbub.googlecode.com

51 Questions?

Make your application real-time with PubSubHubbub. Brett Slatkin May 19th, 2010

Make your application real-time with PubSubHubbub Brett Slatkin May 19th, 2010 View live notes and ask questions about this session on Google Wave http://tinyurl.com/push-io2010 Me http://onebigfluke.com